From owner-freebsd-smp Mon Jul 29 14:30:45 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E6E7E37B400 for ; Mon, 29 Jul 2002 14:30:43 -0700 (PDT) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4273743E3B for ; Mon, 29 Jul 2002 14:30:43 -0700 (PDT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id RAA00347 for ; Mon, 29 Jul 2002 17:30:42 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g6TLUCS12130; Mon, 29 Jul 2002 17:30:12 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15685.46052.696385.530549@grasshopper.cs.duke.edu> Date: Mon, 29 Jul 2002 17:30:12 -0400 (EDT) To: freebsd-smp@freebsd.org Subject: INTR_MPSAFE network drivers X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org What does a network driver need to do to mark itself INTR_MPSAFE? Eg, do I need to get Giant before callng ether_input()? Why are no network drivers marked INTR_MPSAFE currently? Is the kernel not ready for mp safe network drivers? Thanks, Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jul 29 14:53:44 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 00C9D37B400 for ; Mon, 29 Jul 2002 14:53:43 -0700 (PDT) Received: from mail.speakeasy.net (mail13.speakeasy.net [216.254.0.213]) by mx1.FreeBSD.org (Postfix) with ESMTP id 91D8043E65 for ; Mon, 29 Jul 2002 14:53:42 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 7309 invoked from network); 29 Jul 2002 21:53:40 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail13.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 29 Jul 2002 21:53:40 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.5/8.12.5) with ESMTP id g6TLrduR051775; Mon, 29 Jul 2002 17:53:39 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <15685.46052.696385.530549@grasshopper.cs.duke.edu> Date: Mon, 29 Jul 2002 17:53:42 -0400 (EDT) From: John Baldwin To: Andrew Gallatin Subject: RE: INTR_MPSAFE network drivers Cc: freebsd-smp@freebsd.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On 29-Jul-2002 Andrew Gallatin wrote: > > > What does a network driver need to do to mark itself INTR_MPSAFE? > Eg, do I need to get Giant before callng ether_input()? > > Why are no network drivers marked INTR_MPSAFE currently? > Is the kernel not ready for mp safe network drivers? Probably not. Also, I think most of the current driver locks in the network drivers probably aren't right. They really should be replaced with the locks on the ifnet structures associated with each instance when ifnet structure locks are added. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jul 29 14:57: 9 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D59DC37B400 for ; Mon, 29 Jul 2002 14:57:07 -0700 (PDT) Received: from imo-r04.mx.aol.com (imo-r04.mx.aol.com [152.163.225.100]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4480843E4A for ; Mon, 29 Jul 2002 14:57:07 -0700 (PDT) (envelope-from Ajbnet@aol.com) Received: from Ajbnet@aol.com by imo-r04.mx.aol.com (mail_out_v32.21.) id n.ad.20f16dde (3858) for ; Mon, 29 Jul 2002 17:57:01 -0400 (EDT) From: Ajbnet@aol.com Message-ID: Date: Mon, 29 Jul 2002 17:57:00 EDT Subject: SMP compatibilty for a server of mine To: freebsd-smp@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="part1_ad.20f16dde.2a77142c_boundary" X-Mailer: AOL 7.0 for Windows UK sub 10502 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --part1_ad.20f16dde.2a77142c_boundary Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Hi, I have a IBM PC Server 720 based around the Corollary C-Bus II architecture. Is it SMP compatible? Thanks, Alex Burke. --part1_ad.20f16dde.2a77142c_boundary Content-Type: text/html; charset="US-ASCII" Content-Transfer-Encoding: 7bit Hi,

I have a IBM PC Server 720 based around the Corollary C-Bus II architecture. Is it SMP compatible?

Thanks, Alex Burke. --part1_ad.20f16dde.2a77142c_boundary-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jul 29 16: 1: 2 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7106037B400 for ; Mon, 29 Jul 2002 16:01:01 -0700 (PDT) Received: from hawk.mail.pas.earthlink.net (hawk.mail.pas.earthlink.net [207.217.120.22]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2246A43E67 for ; Mon, 29 Jul 2002 16:01:01 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0033.cvx21-bradley.dialup.earthlink.net ([209.179.192.33] helo=mindspring.com) by hawk.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17ZJVW-0005os-00; Mon, 29 Jul 2002 16:00:54 -0700 Message-ID: <3D45C8E5.FB1B7D44@mindspring.com> Date: Mon, 29 Jul 2002 15:59:49 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Ajbnet@aol.com Cc: freebsd-smp@FreeBSD.org Subject: Re: SMP compatibilty for a server of mine References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Ajbnet@aol.com wrote: > > Part 1.1 Type: Plain Text (text/plain) > Encoding: 7bit |I have a IBM PC Server 720 based around the Corollary C-Bus II architecture. | Is it SMP compatible? Does it comply with the Intel Multiprocessing Specification, version 1.4? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jul 29 17: 3:56 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3BE8D37B400; Mon, 29 Jul 2002 17:03:54 -0700 (PDT) Received: from canning.wemm.org (canning.wemm.org [192.203.228.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id 03A5343E3B; Mon, 29 Jul 2002 17:03:50 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by canning.wemm.org (Postfix) with ESMTP id E0D9F2A7D6; Mon, 29 Jul 2002 17:03:45 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: John Baldwin Cc: Andrew Gallatin , freebsd-smp@freebsd.org Subject: Re: INTR_MPSAFE network drivers In-Reply-To: Date: Mon, 29 Jul 2002 17:03:45 -0700 From: Peter Wemm Message-Id: <20020730000345.E0D9F2A7D6@canning.wemm.org> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org John Baldwin wrote: > > On 29-Jul-2002 Andrew Gallatin wrote: > > > > > > What does a network driver need to do to mark itself INTR_MPSAFE? > > Eg, do I need to get Giant before callng ether_input()? > > > > Why are no network drivers marked INTR_MPSAFE currently? > > Is the kernel not ready for mp safe network drivers? > > Probably not. Also, I think most of the current driver locks in > the network drivers probably aren't right. They really should be > replaced with the locks on the ifnet structures associated with > each instance when ifnet structure locks are added. The single beneficial thing we can do that is also really low hanging fruit is to make as many drivers as we can INTR_MPSAFE and move the checking of the hardware interrupt status register outside of giant. This allows us to more efficiently share interrupt between mpsafe and non-mpsafe drivers. The problem is that when an interrupt happens for something that is shared between mpsafe and non-mpsafe drivers, the interrupt is not reenabled again until *all* of the drivers have polled their hardware. And if your mpsafe driver shares with a non-mpsafe driver, then your processing will be held up in a lockstep while the *other* driver blocks on Giant, even if it was *your* interrupt to start with. ie: the other driver will block on giant, check the service routine and most likely then see that nothing is pending and then release giant. This murders your interrupt latency. If every driver checks its status outside of Giant, then this would not be a problem. The catch is that pci_read_config() etc is not mpsafe yet since it uses the shared pci configuration space window with no locking at all... :-] (This should be simple to fix as well, except acpi is involved too now). Cheers, -Peter -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jul 30 6:19:54 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1027F37B400 for ; Tue, 30 Jul 2002 06:19:52 -0700 (PDT) Received: from cod.mar.dfo-mpo.gc.ca (cod.mar.dfo-mpo.gc.ca [142.176.61.253]) by mx1.FreeBSD.org (Postfix) with SMTP id 0489F43E3B for ; Tue, 30 Jul 2002 06:19:51 -0700 (PDT) (envelope-from CampbellG@mar.dfo-mpo.gc.ca) Received: id KAA01188; Tue, 30 Jul 2002 10:18:38 -0300 Received: by gateway id <0H02001ASCWHNJ@bionet.bio.dfo.ca> for freebsd-smp@freebsd.org; Tue, 30 Jul 2002 10:17:05 -0300 (ADT) Received: by gateway id ; Tue, 30 Jul 2002 10:17:03 -0300 Content-return: allowed Date: Tue, 30 Jul 2002 09:01:48 -0300 From: "Campbell, Glenn" Subject: SMP Mainboards To: freebsd-smp@freebsd.org Message-id: <93FC2E7CA4507340BD77D154DD11B8AA1BEDDB@MARSYDE01> MIME-version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-type: text/plain; charset="iso-8859-1" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org I have recently purchased an Intel dual smp board (E7500 Chipset) and two Xeon 2Ghz CPU's with 512k L2 cache each. Does anyone have any experience with these chipsets and BSD? I am looking for optimizations and/or possible problems I may run into. I am going to run BSD 4.6 stable on the system. I was originally torn between this mainboard/cpu config and an MSI Dual Athlon MP board, but after reading Tom's hardware guide review on the E7500 chipset I was sold. The mainboard is specifically the SE7500CW2. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jul 30 6:30:52 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F011D37B4AA; Tue, 30 Jul 2002 06:30:44 -0700 (PDT) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3B19243E3B; Tue, 30 Jul 2002 06:30:44 -0700 (PDT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id JAA17955; Tue, 30 Jul 2002 09:30:43 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g6UDUDo13316; Tue, 30 Jul 2002 09:30:13 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15686.38117.385391.893080@grasshopper.cs.duke.edu> Date: Tue, 30 Jul 2002 09:30:13 -0400 (EDT) To: Peter Wemm Cc: John Baldwin , freebsd-smp@FreeBSD.org Subject: Re: INTR_MPSAFE network drivers In-Reply-To: <20020730000345.E0D9F2A7D6@canning.wemm.org> References: <20020730000345.E0D9F2A7D6@canning.wemm.org> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Peter Wemm writes: > John Baldwin wrote: > > > > On 29-Jul-2002 Andrew Gallatin wrote: > > > > > > > > > What does a network driver need to do to mark itself INTR_MPSAFE? > > > Eg, do I need to get Giant before callng ether_input()? > > > > > > Why are no network drivers marked INTR_MPSAFE currently? > > > Is the kernel not ready for mp safe network drivers? > > > > Probably not. Also, I think most of the current driver locks in > > the network drivers probably aren't right. They really should be > > replaced with the locks on the ifnet structures associated with > > each instance when ifnet structure locks are added. > > The single beneficial thing we can do that is also really low hanging fruit > is to make as many drivers as we can INTR_MPSAFE and move the checking of > the hardware interrupt status register outside of giant. Yes, making my driver INTR_MPSAFE gets back about half of the performance lost when moving from -stable to -current. ;-) > This allows us to more efficiently share interrupt between mpsafe and > non-mpsafe drivers. The problem is that when an interrupt happens for > something that is shared between mpsafe and non-mpsafe drivers, the > interrupt is not reenabled again until *all* of the drivers have polled > their hardware. And if your mpsafe driver shares with a non-mpsafe driver, > then your processing will be held up in a lockstep while the *other* driver > blocks on Giant, even if it was *your* interrupt to start with. ie: the > other driver will block on giant, check the service routine and most likely > then see that nothing is pending and then release giant. > > This murders your interrupt latency. > > If every driver checks its status outside of Giant, then this would not > be a problem. > > The catch is that pci_read_config() etc is not mpsafe yet since it uses the > shared pci configuration space window with no locking at all... :-] > (This should be simple to fix as well, except acpi is involved too now). Ugh. That sounds quite nasty. Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jul 30 8:20:57 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 47C2D37B400; Tue, 30 Jul 2002 08:20:56 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7F04743E4A; Tue, 30 Jul 2002 08:20:55 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.3/8.12.3) with ESMTP id g6UFKfYx055631; Tue, 30 Jul 2002 09:20:49 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Tue, 30 Jul 2002 09:20:34 -0600 (MDT) Message-Id: <20020730.092034.98344815.imp@bsdimp.com> To: peter@wemm.org Cc: jhb@FreeBSD.ORG, gallatin@cs.duke.edu, freebsd-smp@FreeBSD.ORG Subject: Re: INTR_MPSAFE network drivers From: "M. Warner Losh" In-Reply-To: <20020730000345.E0D9F2A7D6@canning.wemm.org> References: <20020730000345.E0D9F2A7D6@canning.wemm.org> X-Mailer: Mew version 2.1 on Emacs 21.2 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org In message: <20020730000345.E0D9F2A7D6@canning.wemm.org> Peter Wemm writes: : The catch is that pci_read_config() etc is not mpsafe yet since it uses the : shared pci configuration space window with no locking at all... :-] : (This should be simple to fix as well, except acpi is involved too now). How many devices check their config space in the ISR, however? Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jul 30 9:30:45 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1ACEB37B400; Tue, 30 Jul 2002 09:30:43 -0700 (PDT) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4BF1743E5E; Tue, 30 Jul 2002 09:30:42 -0700 (PDT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id MAA24072; Tue, 30 Jul 2002 12:30:41 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g6UGU9X13457; Tue, 30 Jul 2002 12:30:09 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15686.48913.150407.307190@grasshopper.cs.duke.edu> Date: Tue, 30 Jul 2002 12:30:09 -0400 (EDT) To: Peter Wemm Cc: John Baldwin , freebsd-smp@FreeBSD.org Subject: Re: INTR_MPSAFE network drivers In-Reply-To: <20020730000345.E0D9F2A7D6@canning.wemm.org> References: <20020730000345.E0D9F2A7D6@canning.wemm.org> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Peter Wemm writes: <..> > The single beneficial thing we can do that is also really low hanging fruit > is to make as many drivers as we can INTR_MPSAFE and move the checking of > the hardware interrupt status register outside of giant. <..> > The catch is that pci_read_config() etc is not mpsafe yet since it uses the > shared pci configuration space window with no locking at all... :-] > (This should be simple to fix as well, except acpi is involved too now). Another catch appears to be that the mbuf codes needs callers of m*_get* to hold Giant, so that it can call kmem_malloc() to expand the mb_map. Here's a panic from my driver running INTR_MPSAFE on a WITNESS-enabled kernel panic: mutex Giant not owned at ../../../vm/vm_kern.c:312 cpuid = 1; lapic.id = 01000000 Debugger("panic") Stopped at Debugger+0x55: xchgl %ebx,in_Debugger.0 db> tr Debugger(c0336e3a,1000000,c0336028,da096b54,c01d7792) at Debugger+0x55 panic(c0336028,c0336192,c034a960,138,da096b88) at panic+0xfd _mtx_assert(c0370980,1,c034a960,138,da096bb4) at _mtx_assert+0xbc kmem_malloc(c1579000,1000,1,1fa,c033969e) at kmem_malloc+0x2d mb_pop_cont(c03a5560,1,c156dee0,283,0) at mb_pop_cont+0xa8 mb_alloc(c03a5560,1,1,0,0) at mb_alloc+0x1ec m_gethdr(1,1,c03a44c0,c4259a14,c1597f00) at m_gethdr+0x34 gm_bsd_ether_rxeof(c4259000,3c,9790,c0372100,0) at gm_bsd_ether_rxeof+0xcd gm_handle_claimed_interrupt(c44e4000,c4541100,c4512400,da096ce0,c455cc1f) at gm_handle_claimed_interrupt+0x180 gm_intr(c44e4000,da096d0c,c01ce9c2,c44e4000,0) at gm_intr+0x28 gm_freebsd_intr(c44e4000,0,c0334294,230,c421bac0) at gm_freebsd_intr+0xb ithread_loop(c4512400,da096d48,c0333f91,355,0) at ithread_loop+0x182 fork_exit(c01ce840,c4512400,da096d48) at fork_exit+0xaf fork_trampoline() at fork_trampoline+0x1a db> What's the prognosis for kmem_malloc() getting out from under Giant? Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jul 30 10:12:30 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DE42037B400; Tue, 30 Jul 2002 10:12:27 -0700 (PDT) Received: from cs.rice.edu (cs.rice.edu [128.42.1.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id 87C8043E31; Tue, 30 Jul 2002 10:12:27 -0700 (PDT) (envelope-from alc@cs.rice.edu) Received: from localhost (localhost [127.0.0.1]) by cs.rice.edu (Postfix) with ESMTP id F25C84A9D2; Tue, 30 Jul 2002 12:12:26 -0500 (CDT) Received: by cs.rice.edu (Postfix, from userid 19572) id 57D004A9B3; Tue, 30 Jul 2002 12:12:26 -0500 (CDT) Date: Tue, 30 Jul 2002 12:12:26 -0500 From: Alan Cox To: Andrew Gallatin Cc: Peter Wemm , John Baldwin , freebsd-smp@FreeBSD.org Subject: Re: INTR_MPSAFE network drivers Message-ID: <20020730171226.GA26599@cs.rice.edu> References: <20020730000345.E0D9F2A7D6@canning.wemm.org> <15686.48913.150407.307190@grasshopper.cs.duke.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <15686.48913.150407.307190@grasshopper.cs.duke.edu> User-Agent: Mutt/1.3.28i X-Virus-Scanned: by AMaViS snapshot-20020300 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Tue, Jul 30, 2002 at 12:30:09PM -0400, Andrew Gallatin wrote: > > Peter Wemm writes: > <..> > > The single beneficial thing we can do that is also really low hanging fruit > > is to make as many drivers as we can INTR_MPSAFE and move the checking of > > the hardware interrupt status register outside of giant. > <..> > > > The catch is that pci_read_config() etc is not mpsafe yet since it uses the > > shared pci configuration space window with no locking at all... :-] > > (This should be simple to fix as well, except acpi is involved too now). > > Another catch appears to be that the mbuf codes needs callers of > m*_get* to hold Giant, so that it can call kmem_malloc() to expand the > mb_map. Remove GIANT_REQUIRED from kmem_malloc() and try again. Seriously. You may or may not trip over another similar assertion, but the odds are that it will work well enough to test. Alan > Here's a panic from my driver running INTR_MPSAFE on a WITNESS-enabled > kernel > > panic: mutex Giant not owned at ../../../vm/vm_kern.c:312 > cpuid = 1; lapic.id = 01000000 > Debugger("panic") > Stopped at Debugger+0x55: xchgl %ebx,in_Debugger.0 > db> tr > Debugger(c0336e3a,1000000,c0336028,da096b54,c01d7792) at Debugger+0x55 > panic(c0336028,c0336192,c034a960,138,da096b88) at panic+0xfd > _mtx_assert(c0370980,1,c034a960,138,da096bb4) at _mtx_assert+0xbc > kmem_malloc(c1579000,1000,1,1fa,c033969e) at kmem_malloc+0x2d > mb_pop_cont(c03a5560,1,c156dee0,283,0) at mb_pop_cont+0xa8 > mb_alloc(c03a5560,1,1,0,0) at mb_alloc+0x1ec > m_gethdr(1,1,c03a44c0,c4259a14,c1597f00) at m_gethdr+0x34 > gm_bsd_ether_rxeof(c4259000,3c,9790,c0372100,0) at gm_bsd_ether_rxeof+0xcd > gm_handle_claimed_interrupt(c44e4000,c4541100,c4512400,da096ce0,c455cc1f) at gm_handle_claimed_interrupt+0x180 > gm_intr(c44e4000,da096d0c,c01ce9c2,c44e4000,0) at gm_intr+0x28 > gm_freebsd_intr(c44e4000,0,c0334294,230,c421bac0) at gm_freebsd_intr+0xb > ithread_loop(c4512400,da096d48,c0333f91,355,0) at ithread_loop+0x182 > fork_exit(c01ce840,c4512400,da096d48) at fork_exit+0xaf > fork_trampoline() at fork_trampoline+0x1a > db> > > > What's the prognosis for kmem_malloc() getting out from under Giant? > > > Drew > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jul 30 10:15:58 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A7B0637B401 for ; Tue, 30 Jul 2002 10:15:56 -0700 (PDT) Received: from maximizemedia.de (legolas.counted.com [66.181.171.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id A8E3443E4A for ; Tue, 30 Jul 2002 10:15:55 -0700 (PDT) (envelope-from fthylmann@maximizemedia.de) Received: from sonic ([80.142.185.115]) by maximizemedia.de ; Tue, 30 Jul 2002 13:17:40 -0400 Message-ID: <014201c237ec$f0a945b0$0201000a@sonic> From: "Fabian Thylmann" To: Subject: SMP Kernel only booting with keyboard? Date: Tue, 30 Jul 2002 19:16:40 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hey, I'm having a strange thing happening here. I have a standard GENERIC kernel with SMP & APIC_IO enabled on an Intel SCB2 board. If no keyboard is connected to the machine, the boot stops at APIC_IO: Testing i8.... interrupt delivery line. If I connect a keyboard it works fine. Can someone tell me why this is happening? Or is it some kind of SCB2 bug? Thanks (hope its ok to post this querstion in this list), Fabian Thylmann To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jul 30 13:41:32 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6456937B400 for ; Tue, 30 Jul 2002 13:41:28 -0700 (PDT) Received: from gull.mail.pas.earthlink.net (gull.mail.pas.earthlink.net [207.217.120.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id 11C6A43E31 for ; Tue, 30 Jul 2002 13:41:28 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from 1cust101.tnt11.sjc4.da.uu.net ([63.42.69.101] helo=mindspring.com) by gull.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17Zdo1-0004jk-00; Tue, 30 Jul 2002 13:41:22 -0700 Message-ID: <3D46F9C0.D1E25132@mindspring.com> Date: Tue, 30 Jul 2002 13:40:32 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Fabian Thylmann Cc: freebsd-smp@freebsd.org Subject: Re: SMP Kernel only booting with keyboard? References: <014201c237ec$f0a945b0$0201000a@sonic> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Fabian Thylmann wrote: > I'm having a strange thing happening here. I have a standard GENERIC kernel > with SMP & APIC_IO enabled on an Intel SCB2 board. If no keyboard is > connected to the machine, the boot stops at APIC_IO: Testing i8.... > interrupt delivery line. > If I connect a keyboard it works fine. > Can someone tell me why this is happening? Or is it some kind of SCB2 bug? > > Thanks (hope its ok to post this querstion in this list), Check your BIOS settings. Normally, this is a symptom of the keyboard appearing to be present even when it's not, which is a BIOS setting. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jul 31 7:22: 0 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4DA6C37B400 for ; Wed, 31 Jul 2002 07:21:59 -0700 (PDT) Received: from out016.verizon.net (out016pub.verizon.net [206.46.170.92]) by mx1.FreeBSD.org (Postfix) with ESMTP id 971F143E31 for ; Wed, 31 Jul 2002 07:21:58 -0700 (PDT) (envelope-from jeff.litchfield@verizon.net) Received: from studio2 ([162.83.148.138]) by out016.verizon.net (InterMail vM.5.01.05.09 201-253-122-126-109-20020611) with SMTP id <20020731142157.SCAB9318.out016.verizon.net@studio2> for ; Wed, 31 Jul 2002 09:21:57 -0500 Message-ID: <000501c2389d$a444d2f0$4ac1fea9@studio2> From: "Jeff" To: Subject: info Date: Wed, 31 Jul 2002 10:22:02 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jul 31 7:46: 8 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5F50A37B40C for ; Wed, 31 Jul 2002 07:46:04 -0700 (PDT) Received: from proton.hexanet.fr (proton.hexanet.fr [81.23.32.33]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8502E43E3B for ; Wed, 31 Jul 2002 07:46:03 -0700 (PDT) (envelope-from c.prevotaux@hexanet.fr) Received: from hexanet.fr (localhost [127.0.0.1]) by proton.hexanet.fr (8.12.3/8.12.3) with SMTP id g6VEk2JU027377 for ; Wed, 31 Jul 2002 16:46:02 +0200 (CEST) (envelope-from c.prevotaux@hexanet.fr) Date: Wed, 31 Jul 2002 16:46:02 +0200 From: Christophe Prevotaux To: freebsd-smp@freebsd.org Subject: NON SMP KERNEL on Dual Proc Machine Message-Id: <20020731164602.58dc9beb.c.prevotaux@hexanet.fr> Organization: HEXANET Sarl X-Mailer: Sylpheed version 0.8.0 (GTK+ 1.2.10; i386-portbld-freebsd4.6) X-NCC-RegID: fr.hexanet Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hi Is it not possible to run a NON SMP KERNEL on a dual Processor machine ? -- -- =============================================================== Christophe Prevotaux Email: c.prevotaux@hexanet.fr HEXANET SARL URL: http://www.hexanet.fr/ Z.A.C Les Charmilles Tel: +33 (0)3 26 79 30 05 3 Allée Thierry Sabine Direct: +33 (0)3 26 61 77 72 BP202 Fax: +33 (0)3 26 79 30 06 51686 Reims Cedex 2 FRANCE HEXANET Network Operation Center =============================================================== To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jul 31 8: 2:20 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1EA4037B400 for ; Wed, 31 Jul 2002 08:02:17 -0700 (PDT) Received: from mail.speakeasy.net (mail17.speakeasy.net [216.254.0.217]) by mx1.FreeBSD.org (Postfix) with ESMTP id 955A543E4A for ; Wed, 31 Jul 2002 08:02:16 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 9095 invoked from network); 31 Jul 2002 15:02:15 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail17.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 31 Jul 2002 15:02:15 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.5/8.12.5) with ESMTP id g6VF2DuR057567; Wed, 31 Jul 2002 11:02:14 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20020731164602.58dc9beb.c.prevotaux@hexanet.fr> Date: Wed, 31 Jul 2002 11:02:14 -0400 (EDT) From: John Baldwin To: Christophe Prevotaux Subject: RE: NON SMP KERNEL on Dual Proc Machine Cc: freebsd-smp@freebsd.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On 31-Jul-2002 Christophe Prevotaux wrote: > > Hi > > Is it not possible to run a NON SMP KERNEL on a > dual Processor machine ? Yes. GENERIFC is a non-SMP kernel for example. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jul 31 10:18: 0 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 690E437B400 for ; Wed, 31 Jul 2002 10:17:52 -0700 (PDT) Received: from maximizemedia.de (legolas.counted.com [66.181.171.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 928A043E3B for ; Wed, 31 Jul 2002 10:17:48 -0700 (PDT) (envelope-from fthylmann@maximizemedia.de) Received: from sonic ([80.142.172.169]) by maximizemedia.de ; Wed, 31 Jul 2002 13:19:30 -0400 Message-ID: <003b01c238b6$6173af50$0201000a@sonic> From: "Fabian Thylmann" To: "Terry Lambert" Cc: References: <014201c237ec$f0a945b0$0201000a@sonic> <3D46F9C0.D1E25132@mindspring.com> Subject: Re: SMP Kernel only booting with keyboard? Date: Wed, 31 Jul 2002 19:18:57 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org I upgraded the bios to the newest version, and there is no option about anything with the keyboard. So is there some other reason for this maybe? Fabian Thylmann ----- Original Message ----- From: "Terry Lambert" To: "Fabian Thylmann" Cc: Sent: Tuesday, July 30, 2002 10:40 PM Subject: Re: SMP Kernel only booting with keyboard? > Fabian Thylmann wrote: > > I'm having a strange thing happening here. I have a standard GENERIC kernel > > with SMP & APIC_IO enabled on an Intel SCB2 board. If no keyboard is > > connected to the machine, the boot stops at APIC_IO: Testing i8.... > > interrupt delivery line. > > If I connect a keyboard it works fine. > > Can someone tell me why this is happening? Or is it some kind of SCB2 bug? > > > > Thanks (hope its ok to post this querstion in this list), > > Check your BIOS settings. > > Normally, this is a symptom of the keyboard appearing to be > present even when it's not, which is a BIOS setting. > > -- Terry > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jul 31 14:22:17 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DCD6737B400 for ; Wed, 31 Jul 2002 14:22:14 -0700 (PDT) Received: from avocet.mail.pas.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 83DBF43E31 for ; Wed, 31 Jul 2002 14:22:14 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0416.cvx40-bradley.dialup.earthlink.net ([216.244.43.161] helo=mindspring.com) by avocet.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17a0v1-0000OR-00; Wed, 31 Jul 2002 14:22:07 -0700 Message-ID: <3D4854CB.281A8CBE@mindspring.com> Date: Wed, 31 Jul 2002 14:21:15 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Fabian Thylmann Cc: freebsd-smp@freebsd.org Subject: Re: SMP Kernel only booting with keyboard? References: <014201c237ec$f0a945b0$0201000a@sonic> <3D46F9C0.D1E25132@mindspring.com> <003b01c238b6$6173af50$0201000a@sonic> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Fabian Thylmann wrote: > I upgraded the bios to the newest version, and there is no option about > anything with the keyboard. > > So is there some other reason for this maybe? You bought the wrong motherboard? Some BIOS require that a keyboard be present. Of these, some will allow you to turn this requirement off in the "advanced setup". Whole companies exist to sell "keyboard dongles": little devices that plug into a keyboard connector, and pretend to be a keyboard well enough to fool these BIOSs. FreeBSD somewhat complicates matters, because it's keyboard probe is rather "aggressive". This also tends to make FreeBSD cause KVM switches, such as the Belkin OmniView line with firmware versions below 1.9 to barf. This is basically to allow probing for the keyboard, which is required to be able to perform a hardware reset line trigger on older systems, and to permit probing for a PS/2 mouse, where the BIOS data is unreliable or the BIOS does not contain explicit mouse support. Nevertheless, Windows doesn't have this problem, even on very old machines. The real answer is to buy only "rack ready" motherboards, if you intend to run without a keyboard present. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jul 31 14:26:56 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4046D37B400; Wed, 31 Jul 2002 14:26:50 -0700 (PDT) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4675B43E65; Wed, 31 Jul 2002 14:26:49 -0700 (PDT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id RAA18171; Wed, 31 Jul 2002 17:26:41 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g6VLQAH15405; Wed, 31 Jul 2002 17:26:10 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15688.22002.772933.316104@grasshopper.cs.duke.edu> Date: Wed, 31 Jul 2002 17:26:10 -0400 (EDT) To: Alan Cox Cc: Peter Wemm , John Baldwin , freebsd-smp@FreeBSD.org Subject: Re: INTR_MPSAFE network drivers In-Reply-To: <20020730171226.GA26599@cs.rice.edu> References: <20020730000345.E0D9F2A7D6@canning.wemm.org> <15686.48913.150407.307190@grasshopper.cs.duke.edu> <20020730171226.GA26599@cs.rice.edu> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Alan Cox writes: > On Tue, Jul 30, 2002 at 12:30:09PM -0400, Andrew Gallatin wrote: > > Another catch appears to be that the mbuf codes needs callers of > > m*_get* to hold Giant, so that it can call kmem_malloc() to expand the > > mb_map. > > Remove GIANT_REQUIRED from kmem_malloc() and try again. Seriously. > You may or may not trip over another similar assertion, but the odds > are that it will work well enough to test. (I also removed the GIANT_REQUIRED in vm_map_lock() as you suggested in private email) It gets a little further, but then trips over GIANT_REQUIRED in vm_page_alloc(): panic(c0335f88,c03360f2,c034bcb2,34a,c1579000) at panic+0xfd _mtx_assert(c03708e0,1,c034bcb2,34a,16a3) at _mtx_assert+0xbc vm_page_alloc(c03cc440,16a3,0,1,c15a2000) at vm_page_alloc+0x3c kmem_malloc(c1579000,1000,1,1fa,c03395fe) at kmem_malloc+0x1b5 mb_pop_cont(c03a54c0,1,c156dee0,283,0) at mb_pop_cont+0xa8 mb_alloc(c03a54c0,1,1,0,0) at mb_alloc+0x1ec m_gethdr(1,1,c03a4420,c4205a7c,c1597400) at m_gethdr+0x34 gm_bsd_ether_rxeof(c4205000,34,52fe,c03720a0,0) at gm_bsd_ether_rxeof+0xcd <...> If I whack that one too, I see this: ../../../kern/kern_synch.c:454: sleeping with "myri" locked from /home/gallatin/gm/gm1/drivers/freebsd/ifnet/gm_bsd_ether_arch.h:112 lock order reversal 1st 0xc4203140 myri (myri) @ /home/gallatin/gm/gm1/drivers/freebsd/ifnet/gm_bsd_ether_arch.h:112 2nd 0xc03708c0 Giant (Giant) @ ../../../vm/vm_object.c:381 panic: mutex Giant not owned at ../../../vm/vm_page.c:597 cpuid = 1; lapic.id = 01000000 Debugger("panic") Stopped at Debugger+0x55: xchgl %ebx,in_Debugger.0 db> show locks exclusive sleep mutex myri r = 0 (0xc4203140) locked @ /home/gallatin/gm/gm1/drivers/freebsd/ifnet/gm_bsd_ether_arch.h:112 db> tr Debugger(c0336d7a,1000000,c0335f68,d7c5bafc,c083aa00) at Debugger+0x55 panic(c0335f68,c03360d2,c034bc92,255,0) at panic+0xfd _mtx_assert(c03708c0,1,c034bc92,255,c0fb6758) at _mtx_assert+0xbc vm_page_insert(c0fb6758,c03cc420,16a4,0,16a4) at vm_page_insert+0x36 vm_page_alloc(c03cc420,16a4,0,1,c15a3000) at vm_page_alloc+0x306 kmem_malloc(c1579000,1000,1,1fa,c03395de) at kmem_malloc+0x1b5 mb_pop_cont(c03a54a0,1,c156c120,283,0) at mb_pop_cont+0xa8 mb_alloc(c03a54a0,1,1,0,0) at mb_alloc+0x1ec m_gethdr(1,1,c03a4400,c4203b4c,c158e100) at m_gethdr+0x34 gm_bsd_ether_rxeof(c4203000,34,7b3c,c0372040,0) at gm_bsd_ether_rxeof+0xcd <...> I get the feeling I should stop now, and revert my tree. FWIW, this is a roughly 1-week old src tree. So I'm missing any improvements you've made in the last week. Would cvs updating from this afternoon make much of a difference? For what its worth, this *seems* to work OK on a non-WITNESS, non-INVARIENTs kernel. Thanks, Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jul 31 19:12:38 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0012637B400; Wed, 31 Jul 2002 19:12:36 -0700 (PDT) Received: from cs.rice.edu (cs.rice.edu [128.42.1.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id A069143E3B; Wed, 31 Jul 2002 19:12:36 -0700 (PDT) (envelope-from alc@cs.rice.edu) Received: from localhost (localhost [127.0.0.1]) by cs.rice.edu (Postfix) with ESMTP id 302684AA0A; Wed, 31 Jul 2002 21:12:36 -0500 (CDT) Received: by cs.rice.edu (Postfix, from userid 19572) id AF6FE4A9ED; Wed, 31 Jul 2002 21:12:35 -0500 (CDT) Date: Wed, 31 Jul 2002 21:12:35 -0500 From: Alan Cox To: Andrew Gallatin Cc: Peter Wemm , John Baldwin , freebsd-smp@FreeBSD.org Subject: Re: INTR_MPSAFE network drivers Message-ID: <20020801021235.GD9934@cs.rice.edu> References: <20020730000345.E0D9F2A7D6@canning.wemm.org> <15686.48913.150407.307190@grasshopper.cs.duke.edu> <20020730171226.GA26599@cs.rice.edu> <15688.22002.772933.316104@grasshopper.cs.duke.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <15688.22002.772933.316104@grasshopper.cs.duke.edu> User-Agent: Mutt/1.3.28i X-Virus-Scanned: by AMaViS snapshot-20020300 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Wed, Jul 31, 2002 at 05:26:10PM -0400, Andrew Gallatin wrote: > > For what its worth, this *seems* to work OK on a non-WITNESS, > non-INVARIENTs kernel. > That's what I would expect. The difficult problem that we'll have with kmem_malloc() and friends stems from legacy drivers that rely on Giant, not a properly locked driver. If a legacy driver sleeps because kmem_malloc() is unable to acquire the mb_map or kernel_map lock, they may wake up to an inconsistent state. Right now, requiring Giant in the top half of the kernel to call kmem_malloc() et al. prevents this. Alan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jul 31 22:44: 9 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4A7B537B400 for ; Wed, 31 Jul 2002 22:44:05 -0700 (PDT) Received: from maximizemedia.de (legolas.counted.com [66.181.171.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9E8F043E6E for ; Wed, 31 Jul 2002 22:44:04 -0700 (PDT) (envelope-from fthylmann@maximizemedia.de) Received: from sonic ([80.142.173.214]) by maximizemedia.de ; Thu, 01 Aug 2002 01:45:46 -0400 Message-ID: <00b601c2391e$a414e4b0$0201000a@sonic> From: "Fabian Thylmann" To: "Terry Lambert" Cc: References: <014201c237ec$f0a945b0$0201000a@sonic> <3D46F9C0.D1E25132@mindspring.com> <003b01c238b6$6173af50$0201000a@sonic> <3D4854CB.281A8CBE@mindspring.com> Subject: Re: SMP Kernel only booting with keyboard? Date: Thu, 1 Aug 2002 07:45:25 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org This is an Intel SCB2 ... Its a Board JUST for servers. That is what confuses me. But I guess I am just goign to have to buy a dongle then. Of course it sounds like a bug in the board, but I was hoping there was a fix that didn't envolve buying a dongle or just leaving a keyboard plugged in. Fabian Thylmann ----- Original Message ----- From: "Terry Lambert" To: "Fabian Thylmann" Cc: Sent: Wednesday, July 31, 2002 11:21 PM Subject: Re: SMP Kernel only booting with keyboard? > Fabian Thylmann wrote: > > I upgraded the bios to the newest version, and there is no option about > > anything with the keyboard. > > > > So is there some other reason for this maybe? > > You bought the wrong motherboard? > > > Some BIOS require that a keyboard be present. Of these, some > will allow you to turn this requirement off in the "advanced > setup". > > Whole companies exist to sell "keyboard dongles": little > devices that plug into a keyboard connector, and pretend to > be a keyboard well enough to fool these BIOSs. > > > FreeBSD somewhat complicates matters, because it's keyboard > probe is rather "aggressive". This also tends to make FreeBSD > cause KVM switches, such as the Belkin OmniView line with > firmware versions below 1.9 to barf. This is basically to > allow probing for the keyboard, which is required to be able > to perform a hardware reset line trigger on older systems, > and to permit probing for a PS/2 mouse, where the BIOS data > is unreliable or the BIOS does not contain explicit mouse > support. Nevertheless, Windows doesn't have this problem, > even on very old machines. > > The real answer is to buy only "rack ready" motherboards, if > you intend to run without a keyboard present. > > -- Terry > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jul 31 23:12:20 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6CAE637B400 for ; Wed, 31 Jul 2002 23:12:19 -0700 (PDT) Received: from snipe.mail.pas.earthlink.net (snipe.mail.pas.earthlink.net [207.217.120.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1352A43E6E for ; Wed, 31 Jul 2002 23:12:19 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0019.cvx21-bradley.dialup.earthlink.net ([209.179.192.19] helo=mindspring.com) by snipe.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17a9Bw-0001lv-00; Wed, 31 Jul 2002 23:12:09 -0700 Message-ID: <3D48D107.74C725AA@mindspring.com> Date: Wed, 31 Jul 2002 23:11:19 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Fabian Thylmann Cc: freebsd-smp@freebsd.org Subject: Re: SMP Kernel only booting with keyboard? References: <014201c237ec$f0a945b0$0201000a@sonic> <3D46F9C0.D1E25132@mindspring.com> <003b01c238b6$6173af50$0201000a@sonic> <3D4854CB.281A8CBE@mindspring.com> <00b601c2391e$a414e4b0$0201000a@sonic> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Fabian Thylmann wrote: > This is an Intel SCB2 ... Its a Board JUST for servers. That is what > confuses me. But I guess I am just goign to have to buy a dongle then. > > Of course it sounds like a bug in the board, but I was hoping there was a > fix that didn't envolve buying a dongle or just leaving a keyboard plugged > in. Usually there is such a fix. In the BIOS settings. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jul 31 23:13:55 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A2D2237B400; Wed, 31 Jul 2002 23:13:52 -0700 (PDT) Received: from snipe.mail.pas.earthlink.net (snipe.mail.pas.earthlink.net [207.217.120.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id 55E7E43E42; Wed, 31 Jul 2002 23:13:52 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0019.cvx21-bradley.dialup.earthlink.net ([209.179.192.19] helo=mindspring.com) by snipe.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17a9DR-0002yj-00; Wed, 31 Jul 2002 23:13:41 -0700 Message-ID: <3D48D163.2213890@mindspring.com> Date: Wed, 31 Jul 2002 23:12:51 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Alan Cox Cc: Andrew Gallatin , Peter Wemm , John Baldwin , freebsd-smp@FreeBSD.org Subject: Re: INTR_MPSAFE network drivers References: <20020730000345.E0D9F2A7D6@canning.wemm.org> <15686.48913.150407.307190@grasshopper.cs.duke.edu> <20020730171226.GA26599@cs.rice.edu> <15688.22002.772933.316104@grasshopper.cs.duke.edu> <20020801021235.GD9934@cs.rice.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Alan Cox wrote: > On Wed, Jul 31, 2002 at 05:26:10PM -0400, Andrew Gallatin wrote: > > > > For what its worth, this *seems* to work OK on a non-WITNESS, > > non-INVARIENTs kernel. > > That's what I would expect. The difficult problem that we'll have > with kmem_malloc() and friends stems from legacy drivers that rely on > Giant, not a properly locked driver. If a legacy driver sleeps because > kmem_malloc() is unable to acquire the mb_map or kernel_map lock, they > may wake up to an inconsistent state. Right now, requiring Giant in > the top half of the kernel to call kmem_malloc() et al. prevents this. Why the heck is a driver calling this anyway? Why the heck is there *anything* in the driver that needs a lock that's not implicit in the fact that it's called to process an interrupt? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Aug 1 4:34:39 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A56AF37B400 for ; Thu, 1 Aug 2002 04:34:34 -0700 (PDT) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1898E43E42 for ; Thu, 1 Aug 2002 04:34:30 -0700 (PDT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id HAA06179; Thu, 1 Aug 2002 07:34:29 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g71BXxP16543; Thu, 1 Aug 2002 07:33:59 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15689.7335.156003.394975@grasshopper.cs.duke.edu> Date: Thu, 1 Aug 2002 07:33:59 -0400 (EDT) To: Terry Lambert Cc: freebsd-smp@FreeBSD.org Subject: Re: INTR_MPSAFE network drivers In-Reply-To: <3D48D163.2213890@mindspring.com> References: <20020730000345.E0D9F2A7D6@canning.wemm.org> <15686.48913.150407.307190@grasshopper.cs.duke.edu> <20020730171226.GA26599@cs.rice.edu> <15688.22002.772933.316104@grasshopper.cs.duke.edu> <20020801021235.GD9934@cs.rice.edu> <3D48D163.2213890@mindspring.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Terry Lambert writes: > Alan Cox wrote: > > On Wed, Jul 31, 2002 at 05:26:10PM -0400, Andrew Gallatin wrote: > > > > > > For what its worth, this *seems* to work OK on a non-WITNESS, > > > non-INVARIENTs kernel. > > > > That's what I would expect. The difficult problem that we'll have > > with kmem_malloc() and friends stems from legacy drivers that rely on > > Giant, not a properly locked driver. If a legacy driver sleeps because > > kmem_malloc() is unable to acquire the mb_map or kernel_map lock, they > > may wake up to an inconsistent state. Right now, requiring Giant in > > the top half of the kernel to call kmem_malloc() et al. prevents this. > > Why the heck is a driver calling this anyway? > > Why the heck is there *anything* in the driver that needs a lock > that's not implicit in the fact that it's called to process an > interrupt? I (and most network drivers) need to call MGETHDR to restock receive buffers from the interrupt handler. When the per-cpu container runs out of mbufs, the system calls mb_pop_cont(). This calls kmem_malloc() to allocate storage for more mbufs: <...> kmem_malloc(c1579000,1000,1,1fa,c03395de) at kmem_malloc+0x1b5 mb_pop_cont(c03a54a0,1,c156c120,283,0) at mb_pop_cont+0xa8 mb_alloc(c03a54a0,1,1,0,0) at mb_alloc+0x1ec m_gethdr(1,1,c03a4400,c4203b4c,c158e100) at m_gethdr+0x34 gm_bsd_ether_rxeof(c4203000,34,7b3c,c0372040,0) at gm_bsd_ether_rxeof+0xcd <...> Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Aug 1 4:53:48 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5F74E37B400; Thu, 1 Aug 2002 04:53:43 -0700 (PDT) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id A4B6643E75; Thu, 1 Aug 2002 04:52:18 -0700 (PDT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id HAA06411; Thu, 1 Aug 2002 07:46:16 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g71Bjkp16557; Thu, 1 Aug 2002 07:45:46 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15689.8042.488037.968605@grasshopper.cs.duke.edu> Date: Thu, 1 Aug 2002 07:45:46 -0400 (EDT) To: Alan Cox Cc: Peter Wemm , John Baldwin , freebsd-smp@FreeBSD.org Subject: Re: INTR_MPSAFE network drivers In-Reply-To: <20020801021235.GD9934@cs.rice.edu> References: <20020730000345.E0D9F2A7D6@canning.wemm.org> <15686.48913.150407.307190@grasshopper.cs.duke.edu> <20020730171226.GA26599@cs.rice.edu> <15688.22002.772933.316104@grasshopper.cs.duke.edu> <20020801021235.GD9934@cs.rice.edu> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Alan Cox writes: > On Wed, Jul 31, 2002 at 05:26:10PM -0400, Andrew Gallatin wrote: > > > > For what its worth, this *seems* to work OK on a non-WITNESS, > > non-INVARIENTs kernel. > > > > That's what I would expect. The difficult problem that we'll have > with kmem_malloc() and friends stems from legacy drivers that rely on > Giant, not a properly locked driver. If a legacy driver sleeps because > kmem_malloc() is unable to acquire the mb_map or kernel_map lock, they > may wake up to an inconsistent state. Right now, requiring Giant in > the top half of the kernel to call kmem_malloc() et al. prevents this. > > Alan A legacy driver will already hold Giant in its interrupt handler by virtue of ithread_loop() acquiring Giant for it: if ((ih->ih_flags & IH_MPSAFE) == 0) mtx_lock(&Giant); ih->ih_handler(ih->ih_argument); if ((ih->ih_flags & IH_MPSAFE) == 0) mtx_unlock(&Giant); So how does requiring Giant in the top half help here? I'm sorry if I'm being dense, but there's a lot about the synchronization requirements in -current that I don't understand. Thanks, Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Aug 1 9:47:37 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E466437B400 for ; Thu, 1 Aug 2002 09:47:34 -0700 (PDT) Received: from atro.pine.nl (atro.pine.nl [213.156.0.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id E1B5043E42 for ; Thu, 1 Aug 2002 09:47:33 -0700 (PDT) (envelope-from mark@atro.pine.nl) Received: by atro.pine.nl (Pine Internet Secure Mailer, from userid 65535) id E466411D001; Thu, 1 Aug 2002 18:47:32 +0200 (MET DST) Date: Thu, 1 Aug 2002 18:47:32 +0200 From: Mark Lastdrager To: freebsd-smp@freebsd.org Subject: booting SMP kernel on single CPU system Message-ID: <20020801164732.GM11068@pine.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i X-NCC-RegID: nl.pine Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hi, I have a 5.0-CURRENT system (build date 25th of May) with SMP kernel. Because my SMP mainboard is under repair I placed the disk in a temporary uniprocessor machine. When I try to boot, the following panic occurs: panic: pmap_bootstrap: no local apic! (non-SMP hardware?) cpuid = 0; lapic.id = 00000000 I understand this problem, but is there some workaround? I don't have a non-SMP kernel on that machine (the kernel.GENERIC is a leftover from FreeBSD 4) and I guess there is no possibily to rebuild the kernel without booting first.. Mark Lastdrager -- Pine Internet BV :: tel. +31-70-3111010 :: fax. +31-70-3111011 PGP 0xFF0EA728 fpr 57D2 CD16 5908 A8F0 9F33 AAA3 AFA0 24EF FF0E A728 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Aug 1 10:49: 1 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 600CD37B400; Thu, 1 Aug 2002 10:48:57 -0700 (PDT) Received: from cs.rice.edu (cs.rice.edu [128.42.1.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0139F43E4A; Thu, 1 Aug 2002 10:48:57 -0700 (PDT) (envelope-from alc@cs.rice.edu) Received: from localhost (localhost [127.0.0.1]) by cs.rice.edu (Postfix) with ESMTP id 7921E4A9F1; Thu, 1 Aug 2002 12:48:56 -0500 (CDT) Received: by cs.rice.edu (Postfix, from userid 19572) id 0516D4A9C7; Thu, 1 Aug 2002 12:48:56 -0500 (CDT) Date: Thu, 1 Aug 2002 12:48:55 -0500 From: Alan Cox To: Andrew Gallatin Cc: Peter Wemm , John Baldwin , freebsd-smp@FreeBSD.org Subject: Re: INTR_MPSAFE network drivers Message-ID: <20020801174855.GE9934@cs.rice.edu> References: <20020730000345.E0D9F2A7D6@canning.wemm.org> <15686.48913.150407.307190@grasshopper.cs.duke.edu> <20020730171226.GA26599@cs.rice.edu> <15688.22002.772933.316104@grasshopper.cs.duke.edu> <20020801021235.GD9934@cs.rice.edu> <15689.8042.488037.968605@grasshopper.cs.duke.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <15689.8042.488037.968605@grasshopper.cs.duke.edu> User-Agent: Mutt/1.3.28i X-Virus-Scanned: by AMaViS snapshot-20020300 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Thu, Aug 01, 2002 at 07:45:46AM -0400, Andrew Gallatin wrote: > > Alan Cox writes: > > On Wed, Jul 31, 2002 at 05:26:10PM -0400, Andrew Gallatin wrote: > > > > > > For what its worth, this *seems* to work OK on a non-WITNESS, > > > non-INVARIENTs kernel. > > > > > > > That's what I would expect. The difficult problem that we'll have > > with kmem_malloc() and friends stems from legacy drivers that rely on > > Giant, not a properly locked driver. If a legacy driver sleeps because > > kmem_malloc() is unable to acquire the mb_map or kernel_map lock, they > > may wake up to an inconsistent state. Right now, requiring Giant in > > the top half of the kernel to call kmem_malloc() et al. prevents this. > > > > Alan > > A legacy driver will already hold Giant in its interrupt handler by > virtue of ithread_loop() acquiring Giant for it: > > if ((ih->ih_flags & IH_MPSAFE) == 0) > mtx_lock(&Giant); > ih->ih_handler(ih->ih_argument); > if ((ih->ih_flags & IH_MPSAFE) == 0) > mtx_unlock(&Giant); > > So how does requiring Giant in the top half help here? You're not being dense. :-) It's subtle. By requiring a top-half thread to hold Giant before acquiring the kernel_map or mb_map locks, we are guaranteeing to the legacy driver that those locks will be free once they get Giant. Consequently, the driver won't be put to sleep in mid operation because it failed to get the kernel_map or mb_map lock. In -STABLE, the same result is achieved via an splvm()/splx() prior to acquiring these locks. (Remember, even -STABLE has vm map locks.) > I'm sorry if I'm being dense, but there's a lot about the > synchronization requirements in -current that I don't understand. Regards, Alan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Aug 1 11:16:30 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C493837B400; Thu, 1 Aug 2002 11:16:27 -0700 (PDT) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9C40A43E5E; Thu, 1 Aug 2002 11:16:23 -0700 (PDT) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id OAA19358; Thu, 1 Aug 2002 14:16:22 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g71IFqD16991; Thu, 1 Aug 2002 14:15:52 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15689.31447.889898.252092@grasshopper.cs.duke.edu> Date: Thu, 1 Aug 2002 14:15:51 -0400 (EDT) To: Alan Cox Cc: Peter Wemm , John Baldwin , freebsd-smp@FreeBSD.org Subject: Re: INTR_MPSAFE network drivers In-Reply-To: <20020801174855.GE9934@cs.rice.edu> References: <20020730000345.E0D9F2A7D6@canning.wemm.org> <15686.48913.150407.307190@grasshopper.cs.duke.edu> <20020730171226.GA26599@cs.rice.edu> <15688.22002.772933.316104@grasshopper.cs.duke.edu> <20020801021235.GD9934@cs.rice.edu> <15689.8042.488037.968605@grasshopper.cs.duke.edu> <20020801174855.GE9934@cs.rice.edu> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Alan Cox writes: > > By requiring a top-half thread to hold Giant before acquiring the > kernel_map or mb_map locks, we are guaranteeing to the legacy driver > that those locks will be free once they get Giant. Consequently, the > driver won't be put to sleep in mid operation because it failed to get > the kernel_map or mb_map lock. Thanks for the spoon feeding. I think I understand now. Next question: Unless I'm mistaken, that means that non-legacy network drivers will not be able to move out from under Giant until there is a way to allocate mbufs which is is certain to never call into the vm system for anything (eg, make M_DONTWAIT really mean M_DONTWAIT). Hmm.. we could have a mclalloc kproc (like -stable does for MCLBYTES > PAGE_SIZE), which would be awakened by the mbuf code in the case that it could not fufill an interrupt time allocation which required calling malloc. I suppose another way to go would be if drivers are prepared to drop their driver locks and get Giant before calling into m_get*. I think most drivers can easily deal with m*get returning NULL, but that it would complicate the locking quite a bit to have to drop driver locks in the middle of the receive path, get Giant, call m_get*, drop Giant, then get driver locks back. Thanks, Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Aug 1 12:21:49 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7020D37B400; Thu, 1 Aug 2002 12:21:48 -0700 (PDT) Received: from cs.rice.edu (cs.rice.edu [128.42.1.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id 150D643E81; Thu, 1 Aug 2002 12:21:48 -0700 (PDT) (envelope-from alc@cs.rice.edu) Received: from localhost (localhost [127.0.0.1]) by cs.rice.edu (Postfix) with ESMTP id B39914A9ED; Thu, 1 Aug 2002 14:21:47 -0500 (CDT) Received: by cs.rice.edu (Postfix, from userid 19572) id 42C304A9C6; Thu, 1 Aug 2002 14:21:47 -0500 (CDT) Date: Thu, 1 Aug 2002 14:21:47 -0500 From: Alan Cox To: Andrew Gallatin Cc: Peter Wemm , John Baldwin , freebsd-smp@FreeBSD.org Subject: Re: INTR_MPSAFE network drivers Message-ID: <20020801192147.GF9934@cs.rice.edu> References: <20020730000345.E0D9F2A7D6@canning.wemm.org> <15686.48913.150407.307190@grasshopper.cs.duke.edu> <20020730171226.GA26599@cs.rice.edu> <15688.22002.772933.316104@grasshopper.cs.duke.edu> <20020801021235.GD9934@cs.rice.edu> <15689.8042.488037.968605@grasshopper.cs.duke.edu> <20020801174855.GE9934@cs.rice.edu> <15689.31447.889898.252092@grasshopper.cs.duke.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <15689.31447.889898.252092@grasshopper.cs.duke.edu> User-Agent: Mutt/1.3.28i X-Virus-Scanned: by AMaViS snapshot-20020300 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Tor Egge and I have discussed refining the notion of a system map to include a subclass of maps that are synchronized by a spin lock. This is not, however, a complete solution. We also have to lock the vm object in which the page is inserted. Alan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Aug 1 13: 5:27 2002 Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 08AFA37B400; Thu, 1 Aug 2002 13:05:24 -0700 (PDT) Received: from swan.mail.pas.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8551B43E65; Thu, 1 Aug 2002 13:05:23 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0503.cvx22-bradley.dialup.earthlink.net ([209.179.199.248] helo=mindspring.com) by swan.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17aMC1-00034s-00; Thu, 01 Aug 2002 13:05:05 -0700 Message-ID: <3D49943E.65FF5AF0@mindspring.com> Date: Thu, 01 Aug 2002 13:04:14 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Andrew Gallatin Cc: Alan Cox , Peter Wemm , John Baldwin , freebsd-smp@FreeBSD.org Subject: Re: INTR_MPSAFE network drivers References: <20020730000345.E0D9F2A7D6@canning.wemm.org> <15686.48913.150407.307190@grasshopper.cs.duke.edu> <20020730171226.GA26599@cs.rice.edu> <15688.22002.772933.316104@grasshopper.cs.duke.edu> <20020801021235.GD9934@cs.rice.edu> <15689.8042.488037.968605@grasshopper.cs.duke.edu> <20020801174855.GE9934@cs.rice.edu> <15689.31447.889898.252092@grasshopper.cs.duke.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Andrew Gallatin wrote: > Unless I'm mistaken, that means that non-legacy network drivers will > not be able to move out from under Giant until there is a way to > allocate mbufs which is is certain to never call into the vm system > for anything (eg, make M_DONTWAIT really mean M_DONTWAIT). Hmm.. we > could have a mclalloc kproc (like -stable does for MCLBYTES > > PAGE_SIZE), which would be awakened by the mbuf code in the case that > it could not fufill an interrupt time allocation which required > calling malloc. > > I suppose another way to go would be if drivers are prepared to drop > their driver locks and get Giant before calling into m_get*. I think > most drivers can easily deal with m*get returning NULL, but that it > would complicate the locking quite a bit to have to drop driver locks > in the middle of the receive path, get Giant, call m_get*, drop Giant, > then get driver locks back. ALL drivers must deal with m_get* returning NULL. It's possible to run out of mbufs entirely, when one runs out of memory, and removing mbufs from the interrupt receive rings is only permitted when there are empty mbufs available to replace the ones you intend to remove (and shove up the stack). Therefore, just returning NULL when the per CPU pool is empty is a valid approach to solving the problem. The only real question is how to refill the per CPU pool in a way that is (effectively) asynchronous, without adding a pool lock. This should be easy to accomplish with a freelist as a linked list, where the update of the list head pointer can be done in an atomic way. It is really very tempting to preallocate kernel mappings for all memory like this that you care about being able to obtain any non-blocking allocations out of; this was the case with the mbuf allocator before it was rewritten, and was the case will all the zalloci zone allocators. I think that the allocation requirements for statistics keeping requiring obtaining a memory manager lock were in fact a mistake; I would rather have statistical statistics (meaning a snapshot, not guaranteed to match reality exactly), and no locks, than the locks that are there now. This was discussed as "future work" when the current code was committed, and I raised this spectre at that time, but it was never committed. One way to face that problem would be to preallocate mappings, but *not* explicitly assign them to a zone. This also resolves the static tuning issues for mbufs and mbuf clusters, which, today, require a precommit of a chunk of KVA space whse size is already known. By creating a mapping pool of KVA space to begin with, *all* memory has the preexisting mappings, which then lets you allocate without creating mappings, and without allocating the mappings to a particular pool (though a rough two tier map for system vs. per-CPU page-pools would be useful). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message