From owner-freebsd-smp Sat Sep 28 08:46:57 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA06522 for smp-outgoing; Sat, 28 Sep 1996 08:46:57 -0700 (PDT) Received: from uno.sat.t.u-tokyo.ac.jp (uno.sat.t.u-tokyo.ac.jp [133.11.70.160]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id IAA04865 for ; Sat, 28 Sep 1996 08:44:13 -0700 (PDT) Received: by uno.sat.t.u-tokyo.ac.jp (8.7.3+2.6Wbeta5/8.7.3) with ESMTP id AAA01959; Sun, 29 Sep 1996 00:43:05 +0900 (JST) To: freebsd-smp@freebsd.org Subject: Re: Tyan S1662 Titan Pro X-Mailer: Mew version 1.06 on Emacs 19.28.1, Mule 2.3 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Sun, 29 Sep 1996 00:43:05 +0900 Message-ID: <1957.843925385@sat.t.u-tokyo.ac.jp> From: Hidetoshi Shimokawa Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, this is my first mail to freebsd-smp. I finally succeeded to run smp-kernel on Tyan S1662 Titan Pro yesterday. It needs some efforts to run stably. As you know, the main problem is that boot cpu id is not 0 but 1. I made some fixes against current smp code (no test1 or test2). - change CPUNBR in mpcore.c - stop cpu0 when smp is inactive in init_main.c - tsleep when boot() is called on cpu0 in kern_shutdown.c these changes are trivial, ad-hoc, and not generic. With only above changes, the kernel had difficuties to change smp_acitve value. It often causes problem in shutdown sequence. I don't know why, but while system is idle, processes is likely to run on cpu0. So boot() in kern_shutdown is usually called on cpu0 and it will do tsleep, but cpu1 never comes up. I guess if boot cpu id is 0, boot() is usually called on boot cpu and it doesn't matter at all. I looked into this problem and I found a potential bug in handing smp_atctive. Here is the code from init_main.c (I added BOOTCPU macro) if ((smp_active == 0) && (cpunumber() != BOOTCPU)) { get_mplock(); printf("cpu#%d freezing; mp_lock: %08x\n",cpunumber(), \ mp_lock); wakeup((caddr_t)&smp_active); rel_mplock(); .... this code is excuted when smp_active = 0, but when smp_active = 0, get_mplock does nothing :-( (look at MPgetlock in mplock.s). I think this causes the problem. my solution is.. - introduce new state smp_active=1, this means only boot cpu is allowed to run, and when boot() is called, smp_active is set to 1. smp_active is 0 only after booting up and once second cpu is lunched, smp_active should change among 1 and 2. I'm not familiar with kernel prgramming, MP and even assembler, I may be misunderstanding something. FYI, host: jp.freebsd.org Tyan S1662 Titan Pro 2 PentiumPro (256KB cache, 233MHz overclocking :-) To build SMP kernel with NetBSD make 168.004s real 238.468s user 70.809s system 184% pmake -j 6 Diffs, have a fun! *** i386/i386/mpcore.s.orig Thu Sep 12 11:40:59 1996 --- i386/i386/mpcore.s Fri Sep 27 02:02:33 1996 *************** *** 159,167 **** movsb ret - /* #define CPUNBR 0x00000000 - */ #ifndef CPUNBR #define CPUNBR 0x01000000 --- 159,165 ---- *** kern/init_main.c.orig Thu Sep 12 11:41:16 1996 --- kern/init_main.c Sat Sep 28 15:40:17 1996 *************** *** 719,731 **** rel_mplock(); while (1) { ! if (smp_active == 0 && cpunumber() != 0) { get_mplock(); printf("cpu#%d freezing; mp_lock: %08x\n", cpunumber(), mp_lock); wakeup((caddr_t)&smp_active); rel_mplock(); ! while (smp_active == 0) { __asm __volatile("" : : : "memory"); } get_mplock(); --- 719,732 ---- rel_mplock(); while (1) { ! #define BOOTCPU 1 ! if ((smp_active == 1) && (cpunumber() != BOOTCPU)) { get_mplock(); printf("cpu#%d freezing; mp_lock: %08x\n", cpunumber(), mp_lock); wakeup((caddr_t)&smp_active); rel_mplock(); ! while (smp_active < 2) { __asm __volatile("" : : : "memory"); } get_mplock(); *** kern/kern_shutdown.c.orig Thu Sep 12 11:41:17 1996 --- kern/kern_shutdown.c Fri Sep 27 22:50:38 1996 *************** *** 171,181 **** #ifdef SMP int c, spins; - smp_active = 0; spins = 100; printf("boot() called on cpu#%d\n", cpunumber()); ! while ((c = cpunumber()) != 0) { if (spins-- < 1) { printf("timeout waiting for cpu zero!\n"); break; --- 171,183 ---- #ifdef SMP int c, spins; spins = 100; + if (smp_active) smp_active = 1; + printf("boot() called on cpu#%d\n", cpunumber()); ! #define BOOTCPU 1 ! while ((c = cpunumber()) != BOOTCPU) { if (spins-- < 1) { printf("timeout waiting for cpu zero!\n"); break; /\ Hidetoshi Shimokawa \/ simokawa@sat.t.u-tokyo.ac.jp PGP public key: finger -l simokawa@sat.t.u-tokyo.ac.jp