Date: Sun, 18 Apr 2010 13:09:59 +0100 From: Rui Paulo <rpaulo@freebsd.org> To: C. Jayachandran <c.jayachandran@gmail.com> Cc: freebsd-mips@freebsd.org Subject: Re: SMP support for XLR processors. Message-ID: <544BD79C-24C0-4015-AE04-F7F8D70CF537@freebsd.org> In-Reply-To: <w2z98a59be81004171540t2f0d5193nca2ec9e2540502e2@mail.gmail.com> References: <w2z98a59be81004171540t2f0d5193nca2ec9e2540502e2@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 17 Apr 2010, at 23:40, C. Jayachandran wrote: > I've a set of initial patches to enable SMP for RMI processors. It > comes up in multi-user with 32 CPUs. I could do buildworld before I > updated to HEAD - with head there is a hang during buildworld which > I'm looking at, but I think the initial work can be checked in. >=20 > Neel, can you have a look at the first two patches - one is to enable > ULE scheduler and the second one is to move platform_init_ap to > slightly later in the initialization sequence. >=20 > The patches are : > 1. mips-ule-support.patch > - Enable ULE scheduler for MIPS >=20 > 2. mips-smp-move-platform.patch > - We need a hook to setup message ring and its interrupts, we use > platform_init_ap now, and move it be called later for XLR >=20 > 3. rmi-prid.patch > Add RMI processor ID prints - right now it prints unknown processor. >=20 > 4. rmi-pcib-fix.patch > XLR pci bridge should be off the IODI bus, currently it is off the > nexus bus which will cause issue when bus_setup_intr on nexus is > called >=20 > 5. cleanup-reorg-cpuid-rge-kx.patch > This set has a many changes rolled up: > - clean up rge driver, remove unused code paths and commented code > (long way to go still) > - xlr_cpu_id(), xlr_core_id() and xlr_thr_id() updated and usage = updated > - fix a bug - we cannot use MIPS_PHYS_TO_KSEG0 on physical addresses > here, I have made changes for using XKPHYS to do the same process. > - the KX bit is enabled and interrupts are disabled before the > physical memory is read, to avoid the KX bit setting from affecting > other code. > - move message ring code to on_chip.c from xlr_machdep.c, now all > message ring code is in on_chip.c >=20 > 6. rmi-xlr-smp.patch > SMP Support for XLR > - mpwait.S added to get the other CPUs out of bootloader code at = startup. > - SMP platform related functions. > - disable shared TLB code for SMP - since threads have different = mappings. >=20 > The patches are also available at > http://sites.google.com/site/cjayachandran/files >=20 > Enabling all 32 threads will need a minor fix in the SMP code, see the > patch subr_smp-fix.patch at the same place. On the netperf XLR, this panics with SCHED_ULE activated: Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights = reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #2 r206791M: Sun Apr 18 12:02:52 UTC 2010 = rpaulo@freebsd-vmware:/usr/home/rpaulo/freebsd/obj/mips/usr/home/rpaulo/fr= eebsd/head/sys/XLR mips real memory =3D 3488608256 (3406844K bytes) avail memory =3D 3395207168 (3237MB) FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs nexus0: <MIPS32 root nexus> iodi0 on nexus0 uart0: <16550 or compatible> on iodi0 uart0: [FILTER] uart0: console (38551,n,8,1) pcib0: <XLR PCI bus> on iodi0 pci0: <PCI bus> on pcib0 pci0: <mass storage> at device 1.0 (no driver attached) pci0: <network, ethernet> at device 3.0 (no driver attached) rge0 on iodi0 rge0: [ITHREAD] rge0: Ethernet address:=20 rge0: [10Mbps] Total Active Core 8 rge1 on iodi0 rge1: [ITHREAD] rge1: Ethernet address: rge1: [1000Mbps] rge2 on iodi0 rge2: [ITHREAD] rge2: Ethernet address:=20 rge2: [10Mbps] rge3 on iodi0 rge3: [ITHREAD] rge3: Ethernet address:=20 rge3: [10Mbps] Enabling MDIO interrupts Timecounter "MIPS32" frequency 66000000 Hz quality 800 Timecounters tick every 1.000 msec bootpc_init: wired to interface 'rge1' Sending DHCP Discover packet from interface rge1 Received DHCP Offer packet on rge1 from 192.168.5.1 (accepted) (no root = path) Sending DHCP Request packet from interface rge1 Sending DHCP Discover packet from interface rge1 DHCP/BOOTP timeout for server 255.255.255.255 DHCP/BOOTP timeout for server 255.255.255.255 DHCP/BOOTP timeout for server 255.255.255.255 Received DHCP Offer packet on rge1 from 192.168.5.1 (accepted) (no root = path) Sending DHCP Request packet from interface rge1 DHCP/BOOTP timeout for server 255.255.255.255 Received DHCP Ack packet on rge1 from 192.168.5.1 (accepted) (got root = path) rge1 at 192.168.5.32 server 192.168.5.1 server name zoo.freebsd.org boot = file /zoo/rpaulo/lama1/boot/kernel/kernel subnet mask 255.255.255.0 root_server 192.168.5.1 rootfs = /zoo/rpaulo/lama1 rootopts nolockd=20 Adjusted interface rge1 panic: mutex sched lock 4 not owned at = /usr/home/rpaulo/freebsd/head/sys/kern/sched_ule.c:2054 cpuid =3D 4 KDB: enter: panic Regards, -- Rui Paulo
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?544BD79C-24C0-4015-AE04-F7F8D70CF537>