Date: Thu, 28 May 1998 17:42:06 -0400 (EDT) From: richard@axil.com To: current@FreeBSD.ORG Subject: FreeBSD on 8-way SMP PPro Message-ID: <199805282142.RAA02782@mint.axil.com>
next in thread | raw e-mail | index | archive | help
I have a hacked version of FreeBSD 3.0-980426-SNAP running on an Axil NX801 (8-way SMP PentiumPro 200MHz, 2 IOAPICs, 4 PCI buses, 7 NCR 53c875 scsi, up to 8GB DRAM and 24*9GB disk, see http://www.axil.com for details). Is this a record for the biggest machine running FreeBSD ? At the moment the hacks are not clean or safe for other hardware, so I won't submit patches yet, but here is a brief description of the changes: 1) i386/pmap.c - pmap_bootstrap() This erroneously uses the physical address io_apic_address[0] instead of io_apic_address[i] for each IOAPIC, so all accesses to the second IOAPIC get mapped to the first, causing considerable confusion ... 2) i386/mp_machdep.c - ap_init() Fixed a race condition - the first cpu to do ap_init(), i.e. the second cpu, would set invltlb_ok = 1; smp_started = 1; But this is not safe until *all* APs have completed ap_init(). This is potentially bad if you have more than 2 CPUs, in practice it seemed to work with 4 CPUs, but usually panic'ed with 6 or 8 CPUs (I didn't try odd numbers). My fix was to remove the "smp_cpus++" at start of ap_init(), and instead put this at the end of ap_init(): asm("lock ; incl _smp_cpus"); /* we're ready to go */ /* * The last cpu to get here enables IPIs etc */ if (smp_cpus >= mp_ncpus) { invltlb_ok = 1; smp_started = 1;/* enable IPI's, tlb shootdown, freezes etc */ smp_active = 1; /* historic */ } With this change it seems to work reliably with 8 CPUs (though I haven't done any stress tests). 3) i386/mpapic.c Various changes to cope with multiple IOAPICs, but my current solution is a kludge so I won't give details here. 4) i386/mp_machdep.c Change INTPIN() to return a hardware interrupt number [0..23] instead of just the IOAPIC pin number, i.e. #if (NAPIC < 2) #define INTPIN(I) (io_apic_ints[(I)].dst_apic_int) #else /* * Fix for multiple IOAPICs Tich 29Apr98 */ int INTPIN(int i) { io_int *ip = &io_apic_ints[i]; int apic, pin; apic = ID_TO_IO(ip->dst_apic_id); pin = ip->dst_apic_int; /* printf("mp_machdep: INTPIN(%d) apic_id %d apic_int %d apic%d.pin%d\n", i, ip->dst_apic_id, ip->dst_apic_int, apic, pin); */ if (apic > 0) pin += NPIN_APIC0; return(pin); } #endif 5) isa/apic_vector.s, isa/apic_ipl.s Various unsatisfactory kludges. I understand changes are afoot in the SMP/IOAPIC code; are there plans to support multiple IOAPICs ? My current solution only supports 16 pins on IOAPIC0 and 8 pins on IOAPIC1 - it looks as though there is code which packs 24 hardware interrupt bits and 7 swi bits into a word, so to support the full 16+24 apic pins requires mapping more than one {ioapic,pin} to each irqNum, which would need changes to INTREN() and INTRDIS(). Let me know if I can be any help. Thanks for the great OS Richard Cownie email: Richard.Cownie@pobox.com DISCLAIMER: I speak for myself, not for Axil Computer To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199805282142.RAA02782>