Date: Fri, 25 Jun 1999 11:02:04 -0700 (PDT) From: Julian Elischer <julian@whistle.com> To: smp@freebsd.org Subject: Re: Call to arms..-SMP (fwd) Message-ID: <Pine.BSF.3.95.990625110011.3553C-100000@current1.whistle.com>
next in thread | raw e-mail | index | archive | help
This message is in MIME format. The first part should be readable text,
while the remaining parts are likely unreadable without MIME-aware tools.
Send mail to mime@docserver.cac.washington.edu for more info.
--0-2067406009-930333724=:3553
Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
Content-ID: <Pine.BSF.3.95.990625110013.3553E@current1.whistle.com>
---------- Forwarded message ----------
Date: Fri, 25 Jun 1999 02:06:13 -0500 (EST)
From: "John S. Dyson" <toor@dyson.iquest.net>
To: Alan Cox <alc@cs.rice.edu>
Cc: luoqi@watermarkgroup.com, dyson@iquest.net, julian@whistle.com
Subject: Re: Call to arms..-SMP
> _cpl is a problem. I noticed this because performance
> went to hell. John, have you thought about this before?
> Can/should _cpl be a per-processor variable?
>
> MPLOCKED incl _cnt+V_SYSCALL
> ECPL_LOCK
> #ifdef CPL_AND_CML
> movl $SWI_AST_MASK,_cml
> #else
> movl $SWI_AST_MASK,_cpl
> #endif
> ECPL_UNLOCK
> call _syscall
>
I have a slightly different scheme for the code that you quote.
Note that mine is NOT necessarily better, so here is my current
version. This begs your question, but I'll try to get back with
you when I am lucid.
Note that the interrupt code that I am using is MUCH shorter
code space wise, once compiled. Again, this isn't responding
to your issue.
First response to your question is that given a global
lock, _cpl should logically be a single variable. CPL itself
is a lame concept (IMO) once you are beyond the actual
interrupt routine itself. IMO, the actual driver should
be a RT scheduleable entity.
It seems that the worst of the benchmark perf is due to
the network code not being able to run concurrently. What
about (for short term a** covering) working the network
code so that it can somehow run concurrently with multiple
socket reads and writes? I don't know what would be
necessary.
BTW, I have put rel_mplock, get_mplock around the vm_page_copy
and vm_page_zero routines also. I don't know if it really
helps, but looks benign. (Note that the pmap manipulations
have to be per-processor for that to work. It is true
for my current pmap code, but I forget about -current.)
John
--0-2067406009-930333724=:3553
Content-Type: *UNKNOWN*/X-SH
Content-ID: <Pine.BSF.3.95.990625110013.3553F@current1.whistle.com>
Content-Description: /sys/i386/i386/exception.s
/*-
* Copyright (c) 1990 The Regents of the University of California.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. All advertising materials mentioning features or use of this software
* must display the following acknowledgement:
* This product includes software developed by the University of
* California, Berkeley and its contributors.
* 4. Neither the name of the University nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* $Id: exception.s,v 1.55 1998/08/10 19:41:07 bde Exp $
*/
#include "npx.h"
#include "opt_vm86.h"
#include <machine/asmacros.h>
#include <machine/ipl.h>
#include <machine/lock.h>
#include <machine/psl.h>
#include <machine/trap.h>
#ifdef SMP
#include <machine/smptests.h> /** CPL_AND_CML, REAL_ */
#endif
#include "assym.s"
#ifndef SMP
#define ECPL_LOCK /* make these nops */
#define ECPL_UNLOCK
#define ICPL_LOCK
#define ICPL_UNLOCK
#define FAST_ICPL_UNLOCK
#define AICPL_LOCK
#define AICPL_UNLOCK
#define AVCPL_LOCK
#define AVCPL_UNLOCK
#endif /* SMP */
#define KCSEL 0x08 /* kernel code selector */
#define KDSEL 0x10 /* kernel data selector */
#define SEL_RPL_MASK 0x0003
#define TRAPF_CS_OFF (13 * 4)
.text
/*****************************************************************************/
/* Trap handling */
/*****************************************************************************/
/*
* Trap and fault vector routines
*/
#define IDTVEC(name) ALIGN_TEXT; .globl __CONCAT(_X,name); __CONCAT(_X,name):
#define TRAP(a) pushl $(a) ; jmp _alltraps
/*
* XXX - debugger traps are now interrupt gates so at least bdb doesn't lose
* control. The sti's give the standard losing behaviour for ddb and kgdb.
*/
#ifdef BDE_DEBUGGER
#define BDBTRAP(name) \
ss ; \
cmpb $0,_bdb_exists ; \
je 1f ; \
testb $SEL_RPL_MASK,4(%esp) ; \
jne 1f ; \
ss ; \
.globl __CONCAT(__CONCAT(bdb_,name),_ljmp); \
__CONCAT(__CONCAT(bdb_,name),_ljmp): \
ljmp $0,$0 ; \
1:
#else
#define BDBTRAP(name)
#endif
#define BPTTRAP(a) testl $PSL_I,4+8(%esp) ; je 1f ; sti ; 1: ; TRAP(a)
MCOUNT_LABEL(user)
MCOUNT_LABEL(btrap)
IDTVEC(div)
pushl $0; TRAP(T_DIVIDE)
IDTVEC(dbg)
BDBTRAP(dbg)
pushl $0; BPTTRAP(T_TRCTRAP)
IDTVEC(nmi)
pushl $0; TRAP(T_NMI)
IDTVEC(bpt)
BDBTRAP(bpt)
pushl $0; BPTTRAP(T_BPTFLT)
IDTVEC(ofl)
pushl $0; TRAP(T_OFLOW)
IDTVEC(bnd)
pushl $0; TRAP(T_BOUND)
IDTVEC(ill)
pushl $0; TRAP(T_PRIVINFLT)
IDTVEC(dna)
pushl $0; TRAP(T_DNA)
IDTVEC(fpusegm)
pushl $0; TRAP(T_FPOPFLT)
IDTVEC(tss)
TRAP(T_TSSFLT)
IDTVEC(missing)
TRAP(T_SEGNPFLT)
IDTVEC(stk)
TRAP(T_STKFLT)
IDTVEC(prot)
TRAP(T_PROTFLT)
IDTVEC(page)
TRAP(T_PAGEFLT)
IDTVEC(mchk)
pushl $0; TRAP(T_MCHK)
IDTVEC(rsvd)
pushl $0; TRAP(T_RESERVED)
IDTVEC(fpu)
#if NNPX > 0
/*
* Handle like an interrupt (except for accounting) so that we can
* call npxintr to clear the error. It would be better to handle
* npx interrupts as traps. This used to be difficult for nested
* interrupts, but now it is fairly easy - mask nested ones the
* same as SWI_AST's.
*/
pushl $0 /* dummy error code */
pushl $0 /* dummy trap type */
pushal
pushl %ds
pushl %es /* now stack frame is a trap frame */
movl $KDSEL,%eax
movl %ax,%ds
movl %ax,%es
FAKE_MCOUNT(12*4(%esp))
#ifdef SMP
MPLOCKED incl _cnt+V_TRAP
FPU_LOCK
ECPL_LOCK
#ifdef CPL_AND_CML
movl _cml,%eax
pushl %eax /* save original cml */
orl $SWI_AST_MASK,%eax
movl %eax,_cml
#else
movl _cpl,%eax
pushl %eax /* save original cpl */
orl $SWI_AST_MASK,%eax
movl %eax,_cpl
#endif /* CPL_AND_CML */
ECPL_UNLOCK
pushl $0 /* dummy unit to finish intr frame */
#else /* SMP */
movl _cpl,%eax
pushl %eax
pushl $0 /* dummy unit to finish intr frame */
incl _cnt+V_TRAP
orl $SWI_AST_MASK,%eax
movl %eax,_cpl
#endif /* SMP */
call _npxintr
incb _intr_nesting_level
MEXITCOUNT
jmp _doreti
#else /* NNPX > 0 */
pushl $0; TRAP(T_ARITHTRAP)
#endif /* NNPX > 0 */
IDTVEC(align)
TRAP(T_ALIGNFLT)
SUPERALIGN_TEXT
.globl _alltraps
_alltraps:
pushal
pushl %ds
pushl %es
alltraps_with_regs_pushed:
movl $KDSEL,%eax
movl %ax,%ds
movl %ax,%es
FAKE_MCOUNT(12*4(%esp))
calltrap:
FAKE_MCOUNT(_btrap) /* init "from" _btrap -> calltrap */
MPLOCKED incl _cnt+V_TRAP
ALIGN_LOCK
ECPL_LOCK
#ifdef CPL_AND_CML
movl _cml,%eax
movl %eax,%ebx /* keep orig. cml here during trap() */
orl $SWI_AST_MASK,%eax
movl %eax,_cml
#else
movl _cpl,%eax
movl %eax,%ebx /* keep orig. cpl here during trap() */
orl $SWI_AST_MASK,%eax
movl %eax,_cpl
#endif
ECPL_UNLOCK
call _trap
/*
* Return via _doreti to handle ASTs. Have to change trap frame
* to interrupt frame.
*/
pushl %ebx /* cpl to restore */
subl $4,%esp /* dummy unit to finish intr frame */
MPLOCKED incb _intr_nesting_level
MEXITCOUNT
jmp _doreti
/*
* Call gate entry for syscall.
* The intersegment call has been set up to specify one dummy parameter.
* This leaves a place to put eflags so that the call frame can be
* converted to a trap frame. Note that the eflags is (semi-)bogusly
* pushed into (what will be) tf_err and then copied later into the
* final spot. It has to be done this way because esp can't be just
* temporarily altered for the pushfl - an interrupt might come in
* and clobber the saved cs/eip.
*/
SUPERALIGN_TEXT
IDTVEC(syscall)
pushfl /* save eflags in tf_err for now */
subl $4,%esp /* skip over tf_trapno */
pushal
pushl %ds
pushl %es
movl $KDSEL,%eax /* switch to kernel segments */
movl %ax,%ds
movl %ax,%es
movl TF_ERR(%esp),%eax /* copy saved eflags to final spot */
movl %eax,TF_EFLAGS(%esp)
movl $7,TF_ERR(%esp) /* sizeof "lcall 7,0" */
FAKE_MCOUNT(12*4(%esp))
MPLOCKED incl _cnt+V_SYSCALL
SYSCALL_LOCK
ECPL_LOCK
#ifdef CPL_AND_CML
movl $SWI_AST_MASK,_cml
#else
movl $SWI_AST_MASK,_cpl
#endif
ECPL_UNLOCK
call _syscall
/*
* Return via _doreti to handle ASTs.
*/
pushl $0 /* cpl to restore */
subl $4,%esp /* dummy unit to finish intr frame */
movb $1,_intr_nesting_level
MEXITCOUNT
jmp _doreti
/*
* Call gate entry for Linux/NetBSD syscall (int 0x80)
*/
SUPERALIGN_TEXT
IDTVEC(int0x80_syscall)
subl $8,%esp /* skip over tf_trapno and tf_err */
pushal
pushl %ds
pushl %es
movl $KDSEL,%eax /* switch to kernel segments */
movl %ax,%ds
movl %ax,%es
movl $2,TF_ERR(%esp) /* sizeof "int 0x80" */
FAKE_MCOUNT(12*4(%esp))
MPLOCKED incl _cnt+V_SYSCALL
ALTSYSCALL_LOCK
ECPL_LOCK
#ifdef CPL_AND_CML
movl $SWI_AST_MASK,_cml
#else
movl $SWI_AST_MASK,_cpl
#endif
ECPL_UNLOCK
call _syscall
/*
* Return via _doreti to handle ASTs.
*/
pushl $0 /* cpl to restore */
subl $4,%esp /* dummy unit to finish intr frame */
movb $1,_intr_nesting_level
MEXITCOUNT
jmp _doreti
ENTRY(fork_trampoline)
call _spl0
movl _curproc,%eax
addl $P_SWITCHTIME,%eax
movl _switchtime,%ecx
testl %ecx,%ecx
jne 1f
/* XXX unreachable? */
pushl %eax
call _microuptime
popl %eax
jmp 2f
1:
movl %ecx,(%eax)
movl _switchtime+4,%edx
movl %edx,4(%eax)
2:
/*
* cpu_set_fork_handler intercepts this function call to
* have this call a non-return function to stay in kernel mode.
* initproc has its own fork handler, but it does return.
*/
pushl %ebx /* arg1 */
call %esi /* function */
addl $4,%esp
/* cut from syscall */
/*
* Return via _doreti to handle ASTs.
*/
pushl $0 /* cpl to restore */
subl $4,%esp /* dummy unit to finish intr frame */
movb $1,_intr_nesting_level
MEXITCOUNT
jmp _doreti
#ifdef VM86
/*
* Include vm86 call routines, which want to call _doreti.
*/
#include "i386/i386/vm86bios.s"
#endif /* VM86 */
/*
* Include what was once config+isa-dependent code.
* XXX it should be in a stand-alone file. It's still icu-dependent and
* belongs in i386/isa.
*/
#include "i386/isa/vector.s"
/*
* Include what was once icu-dependent code.
* XXX it should be merged into this file (also move the definition of
* imen to vector.s or isa.c).
* Before including it, set up a normal asm environment so that vector.s
* doesn't have to know that stuff is included after it.
*/
.data
ALIGN_DATA
.text
SUPERALIGN_TEXT
#include "i386/isa/ipl.s"
--0-2067406009-930333724=:3553
Content-Type: *UNKNOWN*/X-SH
Content-ID: <Pine.BSF.3.95.990625110013.3553G@current1.whistle.com>
Content-Description: /sys/i386/isa/apic_vector.s
/*
* from: vector.s, 386BSD 0.1 unknown origin
* $Id: apic_vector.s,v 1.34 1998/09/06 22:41:41 tegge Exp $
*/
#include <machine/apic.h>
#include <machine/smp.h>
#include "i386/isa/intr_machdep.h"
#ifdef FAST_SIMPLELOCK
#define GET_FAST_INTR_LOCK \
pushl $_fast_intr_lock ; /* address of lock */ \
call _s_lock ; /* MP-safe */ \
addl $4,%esp
#define REL_FAST_INTR_LOCK \
pushl $_fast_intr_lock ; /* address of lock */ \
call _s_unlock ; /* MP-safe */ \
addl $4,%esp
#else /* FAST_SIMPLELOCK */
#define GET_FAST_INTR_LOCK \
call _get_isrlock
#define REL_FAST_INTR_LOCK \
pushl $_mp_lock ; /* GIANT_LOCK */ \
call _MPrellock ; \
add $4, %esp
#endif /* FAST_SIMPLELOCK */
/* convert an absolute IRQ# into a bitmask */
#define IRQ_BIT(irq_num) (1 << (irq_num))
/* make an index into the IO APIC from the IRQ# */
#define REDTBL_IDX(irq_num) (0x10 + ((irq_num) * 2))
/*
* Macros for interrupt interrupt entry, call to handler, and exit.
*/
#ifdef FAST_WITHOUTCPL
/*
*/
#define FAST_INTR(irq_num, vec_name) \
.text ; \
SUPERALIGN_TEXT ; \
IDTVEC(vec_name) ; \
pushl %eax ; /* save only call-used registers */ \
pushl %ecx ; \
pushl %edx ; \
pushl %ds ; \
MAYBE_PUSHL_ES ; \
movl $KDSEL,%eax ; \
movl %ax,%ds ; \
MAYBE_MOVW_AX_ES ; \
FAKE_MCOUNT((4+ACTUALLY_PUSHED)*4(%esp)) ; \
pushl _intr_unit + (irq_num) * 4 ; \
GET_FAST_INTR_LOCK ; \
call *_intr_handler + (irq_num) * 4 ; /* do the work ASAP */ \
REL_FAST_INTR_LOCK ; \
addl $4, %esp ; \
movl $0, lapic_eoi ; \
lock ; \
incl _cnt+V_INTR ; /* book-keeping can wait */ \
movl _intr_countp + (irq_num) * 4, %eax ; \
lock ; \
incl (%eax) ; \
MEXITCOUNT ; \
MAYBE_POPL_ES ; \
popl %ds ; \
popl %edx ; \
popl %ecx ; \
popl %eax ; \
iret
#else /* FAST_WITHOUTCPL */
#define FAST_INTR(irq_num, vec_name) \
.text ; \
SUPERALIGN_TEXT ; \
IDTVEC(vec_name) ; \
pushl %eax ; /* save only call-used registers */ \
pushl %ecx ; \
pushl %edx ; \
pushl %ds ; \
MAYBE_PUSHL_ES ; \
movl $KDSEL, %eax ; \
movl %ax, %ds ; \
MAYBE_MOVW_AX_ES ; \
FAKE_MCOUNT((4+ACTUALLY_PUSHED)*4(%esp)) ; \
GET_FAST_INTR_LOCK ; \
pushl _intr_unit + (irq_num) * 4 ; \
call *_intr_handler + (irq_num) * 4 ; /* do the work ASAP */ \
addl $4, %esp ; \
movl $0, lapic_eoi ; \
lock ; \
incl _cnt+V_INTR ; /* book-keeping can wait */ \
movl _intr_countp + (irq_num) * 4,%eax ; \
lock ; \
incl (%eax) ; \
movl _cpl, %eax ; /* unmasking pending HWIs or SWIs? */ \
notl %eax ; \
andl _ipending, %eax ; \
jne 2f ; /* yes, maybe handle them */ \
1: ; \
MEXITCOUNT ; \
REL_FAST_INTR_LOCK ; \
MAYBE_POPL_ES ; \
popl %ds ; \
popl %edx ; \
popl %ecx ; \
popl %eax ; \
iret ; \
; \
ALIGN_TEXT ; \
2: ; \
cmpb $3, _intr_nesting_level ; /* enough stack? */ \
jae 1b ; /* no, return */ \
movl _cpl, %eax ; \
/* XXX next line is probably unnecessary now. */ \
movl $HWI_MASK|SWI_MASK, _cpl ; /* limit nesting ... */ \
lock ; \
incb _intr_nesting_level ; /* ... really limit it ... */ \
sti ; /* to do this as early as possible */ \
MAYBE_POPL_ES ; /* discard most of thin frame ... */ \
popl %ecx ; /* ... original %ds ... */ \
popl %edx ; \
xchgl %eax, 4(%esp) ; /* orig %eax; save cpl */ \
pushal ; /* build fat frame (grrr) ... */ \
pushl %ecx ; /* ... actually %ds ... */ \
pushl %es ; \
movl $KDSEL, %eax ; \
movl %ax, %es ; \
movl (2+8+0)*4(%esp), %ecx ; /* %ecx from thin frame ... */ \
movl %ecx, (2+6)*4(%esp) ; /* ... to fat frame ... */ \
movl (2+8+1)*4(%esp), %eax ; /* ... cpl from thin frame */ \
pushl %eax ; \
subl $4, %esp ; /* junk for unit number */ \
MEXITCOUNT ; \
jmp _doreti
#endif /** FAST_WITHOUTCPL */
/*
*
*/
#define PUSH_FRAME \
pushl $0 ; /* dummy error code */ \
pushl $0 ; /* dummy trap type */ \
pushal ; \
pushl %ds ; /* save data and extra segments ... */ \
pushl %es
#define POP_FRAME \
popl %es ; \
popl %ds ; \
popal ; \
addl $4+4,%esp
#define IOAPICADDR(irq_num) CNAME(int_to_apicintpin) + 16 * (irq_num) + 8
#define REDIRIDX(irq_num) CNAME(int_to_apicintpin) + 16 * (irq_num) + 12
#define MASK_IRQ(irq_num) \
IMASK_LOCK ; /* into critical reg */ \
testl $IRQ_BIT(irq_num), _apic_imen ; \
jne 7f ; /* masked, don't mask */ \
orl $IRQ_BIT(irq_num), _apic_imen ; /* set the mask bit */ \
movl IOAPICADDR(irq_num), %ecx ; /* ioapic addr */ \
movl REDIRIDX(irq_num), %eax ; /* get the index */ \
movl %eax, (%ecx) ; /* write the index */ \
movl IOAPIC_WINDOW(%ecx), %eax ; /* current value */ \
orl $IOART_INTMASK, %eax ; /* set the mask */ \
movl %eax, IOAPIC_WINDOW(%ecx) ; /* new value */ \
7: ; /* already masked */ \
IMASK_UNLOCK
/*
* Test to see whether we are handling an edge or level triggered INT.
* Level-triggered INTs must still be masked as we don't clear the source,
* and the EOI cycle would cause redundant INTs to occur.
*/
#define MASK_LEVEL_IRQ(irq_num) \
testl $IRQ_BIT(irq_num), _apic_pin_trigger ; \
jz 9f ; /* edge, don't mask */ \
MASK_IRQ(irq_num) ; \
9:
#ifdef APIC_INTR_REORDER
#define EOI_IRQ(irq_num) \
movl _apic_isrbit_location + 8 * (irq_num), %eax ; \
movl (%eax), %eax ; \
testl _apic_isrbit_location + 4 + 8 * (irq_num), %eax ; \
jz 9f ; /* not active */ \
movl $0, lapic_eoi ; \
APIC_ITRACE(apic_itrace_eoi, irq_num, APIC_ITRACE_EOI) ; \
9:
#else
#define EOI_IRQ(irq_num) \
testl $IRQ_BIT(irq_num), lapic_isr1; \
jz 9f ; /* not active */ \
movl $0, lapic_eoi; \
APIC_ITRACE(apic_itrace_eoi, irq_num, APIC_ITRACE_EOI) ; \
9:
#endif
/*
* Test to see if the source is currntly masked, clear if so.
*/
#define UNMASK_IRQ(irq_num) \
IMASK_LOCK ; /* into critical reg */ \
testl $IRQ_BIT(irq_num), _apic_imen ; \
je 7f ; /* bit clear, not masked */ \
andl $~IRQ_BIT(irq_num), _apic_imen ;/* clear mask bit */ \
movl IOAPICADDR(irq_num),%ecx ; /* ioapic addr */ \
movl REDIRIDX(irq_num), %eax ; /* get the index */ \
movl %eax,(%ecx) ; /* write the index */ \
movl IOAPIC_WINDOW(%ecx),%eax ; /* current value */ \
andl $~IOART_INTMASK,%eax ; /* clear the mask */ \
movl %eax,IOAPIC_WINDOW(%ecx) ; /* new value */ \
7: ; \
IMASK_UNLOCK
#ifdef INTR_SIMPLELOCK
#define ENLOCK
#define DELOCK
#define LATELOCK call _get_isrlock
#else
#define ENLOCK \
ISR_TRYLOCK ; /* XXX this is going away... */ \
testl %eax, %eax ; /* did we get it? */ \
jz 3f
#define DELOCK ISR_RELLOCK
#define LATELOCK
#endif
#ifdef APIC_INTR_DIAGNOSTIC
#ifdef APIC_INTR_DIAGNOSTIC_IRQ
log_intr_event:
pushf
cli
pushl $CNAME(apic_itrace_debuglock)
call _s_lock_np
addl $4, %esp
movl CNAME(apic_itrace_debugbuffer_idx), %ecx
andl $32767, %ecx
movl _cpuid, %eax
shll $8, %eax
orl 8(%esp), %eax
movw %ax, CNAME(apic_itrace_debugbuffer)(,%ecx,2)
incl %ecx
andl $32767, %ecx
movl %ecx, CNAME(apic_itrace_debugbuffer_idx)
pushl $CNAME(apic_itrace_debuglock)
call _s_unlock_np
addl $4, %esp
popf
ret
#define APIC_ITRACE(name, irq_num, id) \
lock ; /* MP-safe */ \
incl CNAME(name) + (irq_num) * 4 ; \
pushl %eax ; \
pushl %ecx ; \
pushl %edx ; \
movl $(irq_num), %eax ; \
cmpl $APIC_INTR_DIAGNOSTIC_IRQ, %eax ; \
jne 7f ; \
pushl $id ; \
call log_intr_event ; \
addl $4, %esp ; \
7: ; \
popl %edx ; \
popl %ecx ; \
popl %eax
#else
#define APIC_ITRACE(name, irq_num, id) \
lock ; /* MP-safe */ \
incl CNAME(name) + (irq_num) * 4
#endif
#define APIC_ITRACE_ENTER 1
#define APIC_ITRACE_EOI 2
#define APIC_ITRACE_TRYISRLOCK 3
#define APIC_ITRACE_GOTISRLOCK 4
#define APIC_ITRACE_ENTER2 5
#define APIC_ITRACE_LEAVE 6
#define APIC_ITRACE_UNMASK 7
#define APIC_ITRACE_ACTIVE 8
#define APIC_ITRACE_MASKED 9
#define APIC_ITRACE_NOISRLOCK 10
#define APIC_ITRACE_MASKED2 11
#define APIC_ITRACE_SPLZ 12
#define APIC_ITRACE_DORETI 13
#else
#define APIC_ITRACE(name, irq_num, id)
#endif
#ifdef CPL_AND_CML
#define INTR(irq_num, vec_name) \
.text ; \
SUPERALIGN_TEXT ; \
/* _XintrNN: entry point used by IDT/HWIs & splz_unpend via _vec[]. */ \
IDTVEC(vec_name) ; \
PUSH_FRAME ; \
movl $KDSEL, %eax ; /* reload with kernel's data segment */ \
movl %ax, %ds ; \
movl %ax, %es ; \
; \
APIC_ITRACE(apic_itrace_enter, irq_num, APIC_ITRACE_ENTER) ; \
lock ; /* MP-safe */ \
btsl $(irq_num), iactive ; /* lazy masking */ \
jc 1f ; /* already active */ \
; \
MASK_LEVEL_IRQ(irq_num) ; \
EOI_IRQ(irq_num) ; \
0: ; \
APIC_ITRACE(apic_itrace_tryisrlock, irq_num, APIC_ITRACE_TRYISRLOCK) ;\
ENLOCK ; \
; \
APIC_ITRACE(apic_itrace_gotisrlock, irq_num, APIC_ITRACE_GOTISRLOCK) ;\
AVCPL_LOCK ; /* MP-safe */ \
testl $IRQ_BIT(irq_num), _cpl ; \
jne 2f ; /* this INT masked */ \
testl $IRQ_BIT(irq_num), _cml ; \
jne 2f ; /* this INT masked */ \
orl $IRQ_BIT(irq_num), _cil ; \
AVCPL_UNLOCK ; \
; \
incb _intr_nesting_level ; \
; \
/* entry point used by doreti_unpend for HWIs. */ \
__CONCAT(Xresume,irq_num): ; \
FAKE_MCOUNT(12*4(%esp)) ; /* XXX avoid dbl cnt */ \
lock ; incl _cnt+V_INTR ; /* tally interrupts */ \
movl _intr_countp + (irq_num) * 4, %eax ; \
lock ; incl (%eax) ; \
; \
AVCPL_LOCK ; /* MP-safe */ \
movl _cml, %eax ; \
pushl %eax ; \
orl _intr_mask + (irq_num) * 4, %eax ; \
movl %eax, _cml ; \
AVCPL_UNLOCK ; \
; \
pushl _intr_unit + (irq_num) * 4 ; \
incl _inside_intr ; \
APIC_ITRACE(apic_itrace_enter2, irq_num, APIC_ITRACE_ENTER2) ; \
sti ; \
call *_intr_handler + (irq_num) * 4 ; \
cli ; \
APIC_ITRACE(apic_itrace_leave, irq_num, APIC_ITRACE_LEAVE) ; \
decl _inside_intr ; \
; \
lock ; andl $~IRQ_BIT(irq_num), iactive ; \
lock ; andl $~IRQ_BIT(irq_num), _cil ; \
UNMASK_IRQ(irq_num) ; \
APIC_ITRACE(apic_itrace_unmask, irq_num, APIC_ITRACE_UNMASK) ; \
sti ; /* doreti repeats cli/sti */ \
MEXITCOUNT ; \
LATELOCK ; \
jmp _doreti ; \
; \
ALIGN_TEXT ; \
1: ; /* active */ \
APIC_ITRACE(apic_itrace_active, irq_num, APIC_ITRACE_ACTIVE) ; \
MASK_IRQ(irq_num) ; \
EOI_IRQ(irq_num) ; \
AVCPL_LOCK ; /* MP-safe */ \
orl $IRQ_BIT(irq_num), _ipending ; \
AVCPL_UNLOCK ; \
lock ; \
btsl $(irq_num), iactive ; /* still active */ \
jnc 0b ; /* retry */ \
POP_FRAME ; \
iret ; \
; \
ALIGN_TEXT ; \
2: ; /* masked by cpl|cml */ \
APIC_ITRACE(apic_itrace_masked, irq_num, APIC_ITRACE_MASKED) ; \
orl $IRQ_BIT(irq_num), _ipending ; \
AVCPL_UNLOCK ; \
DELOCK ; /* XXX this is going away... */ \
POP_FRAME ; \
iret ; \
ALIGN_TEXT ; \
3: ; /* other cpu has isr lock */ \
APIC_ITRACE(apic_itrace_noisrlock, irq_num, APIC_ITRACE_NOISRLOCK) ;\
AVCPL_LOCK ; /* MP-safe */ \
orl $IRQ_BIT(irq_num), _ipending ; \
testl $IRQ_BIT(irq_num), _cpl ; \
jne 4f ; /* this INT masked */ \
testl $IRQ_BIT(irq_num), _cml ; \
jne 4f ; /* this INT masked */ \
orl $IRQ_BIT(irq_num), _cil ; \
AVCPL_UNLOCK ; \
call forward_irq ; /* forward irq to lock holder */ \
POP_FRAME ; /* and return */ \
iret ; \
ALIGN_TEXT ; \
4: ; /* blocked */ \
APIC_ITRACE(apic_itrace_masked2, irq_num, APIC_ITRACE_MASKED2) ;\
AVCPL_UNLOCK ; \
POP_FRAME ; /* and return */ \
iret
#else /* CPL_AND_CML */
#define INTR(irq_num, vec_name) \
.text ; \
SUPERALIGN_TEXT ; \
/* _XintrNN: entry point used by IDT/HWIs & splz_unpend via _vec[]. */ \
IDTVEC(vec_name) ; \
PUSH_FRAME ; \
movl $KDSEL, %eax ; /* reload with kernel's data segment */ \
movl %ax, %ds ; \
movl %ax, %es ; \
; \
APIC_ITRACE(apic_itrace_enter, irq_num, APIC_ITRACE_ENTER) ; \
lock ; /* MP-safe */ \
btsl $(irq_num), iactive ; /* lazy masking */ \
jc 1f ; /* already active */ \
; \
MASK_LEVEL_IRQ(irq_num) ; \
EOI_IRQ(irq_num) ; \
0: ; \
APIC_ITRACE(apic_itrace_tryisrlock, irq_num, APIC_ITRACE_TRYISRLOCK) ;\
ISR_TRYLOCK ; /* XXX this is going away... */ \
testl %eax, %eax ; /* did we get it? */ \
jz 3f ; /* no */ \
; \
APIC_ITRACE(apic_itrace_gotisrlock, irq_num, APIC_ITRACE_GOTISRLOCK) ;\
AVCPL_LOCK ; /* MP-safe */ \
testl $IRQ_BIT(irq_num), _cpl ; \
jne 2f ; /* this INT masked */ \
AVCPL_UNLOCK ; \
; \
incb _intr_nesting_level ; \
; \
/* entry point used by doreti_unpend for HWIs. */ \
__CONCAT(Xresume,irq_num): ; \
FAKE_MCOUNT(12*4(%esp)) ; /* XXX avoid dbl cnt */ \
lock ; incl _cnt+V_INTR ; /* tally interrupts */ \
movl _intr_countp + (irq_num) * 4, %eax ; \
lock ; incl (%eax) ; \
; \
AVCPL_LOCK ; /* MP-safe */ \
movl _cpl, %eax ; \
pushl %eax ; \
orl _intr_mask + (irq_num) * 4, %eax ; \
movl %eax, _cpl ; \
andl $~IRQ_BIT(irq_num), _ipending ; \
AVCPL_UNLOCK ; \
; \
pushl _intr_unit + (irq_num) * 4 ; \
APIC_ITRACE(apic_itrace_enter2, irq_num, APIC_ITRACE_ENTER2) ; \
sti ; \
call *_intr_handler + (irq_num) * 4 ; \
cli ; \
APIC_ITRACE(apic_itrace_leave, irq_num, APIC_ITRACE_LEAVE) ; \
; \
lock ; andl $~IRQ_BIT(irq_num), iactive ; \
UNMASK_IRQ(irq_num) ; \
APIC_ITRACE(apic_itrace_unmask, irq_num, APIC_ITRACE_UNMASK) ; \
sti ; /* doreti repeats cli/sti */ \
MEXITCOUNT ; \
jmp _doreti ; \
; \
ALIGN_TEXT ; \
1: ; /* active */ \
APIC_ITRACE(apic_itrace_active, irq_num, APIC_ITRACE_ACTIVE) ; \
MASK_IRQ(irq_num) ; \
EOI_IRQ(irq_num) ; \
AVCPL_LOCK ; /* MP-safe */ \
orl $IRQ_BIT(irq_num), _ipending ; \
AVCPL_UNLOCK ; \
lock ; \
btsl $(irq_num), iactive ; /* still active */ \
jnc 0b ; /* retry */ \
POP_FRAME ; \
iret ; /* XXX: iactive bit might be 0 now */ \
ALIGN_TEXT ; \
2: ; /* masked by cpl, leave iactive set */ \
APIC_ITRACE(apic_itrace_masked, irq_num, APIC_ITRACE_MASKED) ; \
orl $IRQ_BIT(irq_num), _ipending ; \
AVCPL_UNLOCK ; \
ISR_RELLOCK ; /* XXX this is going away... */ \
POP_FRAME ; \
iret ; \
ALIGN_TEXT ; \
3: ; /* other cpu has isr lock */ \
APIC_ITRACE(apic_itrace_noisrlock, irq_num, APIC_ITRACE_NOISRLOCK) ;\
AVCPL_LOCK ; /* MP-safe */ \
orl $IRQ_BIT(irq_num), _ipending ; \
testl $IRQ_BIT(irq_num), _cpl ; \
jne 4f ; /* this INT masked */ \
AVCPL_UNLOCK ; \
call forward_irq ; /* forward irq to lock holder */ \
POP_FRAME ; /* and return */ \
iret ; \
ALIGN_TEXT ; \
4: ; /* blocked */ \
APIC_ITRACE(apic_itrace_masked2, irq_num, APIC_ITRACE_MASKED2) ;\
AVCPL_UNLOCK ; \
POP_FRAME ; /* and return */ \
iret
#endif /* CPL_AND_CML */
/*
* Handle "spurious INTerrupts".
* Notes:
* This is different than the "spurious INTerrupt" generated by an
* 8259 PIC for missing INTs. See the APIC documentation for details.
* This routine should NOT do an 'EOI' cycle.
*/
.text
SUPERALIGN_TEXT
.globl _Xspuriousint
_Xspuriousint:
/* No EOI cycle used here */
iret
/*
* Handle TLB shootdowns.
*/
.text
SUPERALIGN_TEXT
.globl _Xinvltlb
_Xinvltlb:
pushl %eax
#ifdef COUNT_XINVLTLB_HITS
ss
movl _cpuid, %eax
ss
incl _xhits(,%eax,4)
#endif /* COUNT_XINVLTLB_HITS */
movl %cr3, %eax /* invalidate the TLB */
movl %eax, %cr3
ss /* stack segment, avoid %ds load */
movl $0, lapic_eoi /* End Of Interrupt to APIC */
popl %eax
iret
#ifdef BETTER_CLOCK
/*
* Executed by a CPU when it receives an Xcpucheckstate IPI from another CPU,
*
* - Stores current cpu state in checkstate_cpustate[cpuid]
* 0 == user, 1 == sys, 2 == intr
* - Stores current process in checkstate_curproc[cpuid]
*
* - Signals its receipt by setting bit cpuid in checkstate_probed_cpus.
*
* stack: 0 -> ds, 4 -> ebx, 8 -> eax, 12 -> eip, 16 -> cs, 20 -> eflags
*/
.text
SUPERALIGN_TEXT
.globl _Xcpucheckstate
.globl _checkstate_cpustate
.globl _checkstate_curproc
.globl _checkstate_pc
_Xcpucheckstate:
pushl %eax
pushl %ebx
pushl %ds /* save current data segment */
movl $KDSEL, %eax
movl %ax, %ds /* use KERNEL data segment */
movl $0, lapic_eoi /* End Of Interrupt to APIC */
movl $0, %ebx
movl 16(%esp), %eax
andl $3, %eax
cmpl $3, %eax
je 1f
#ifdef VM86
testl $PSL_VM, 20(%esp)
jne 1f
#endif
incl %ebx /* system or interrupt */
#ifdef CPL_AND_CML
cmpl $0, _inside_intr
je 1f
incl %ebx /* interrupt */
#endif
1:
movl _cpuid, %eax
movl %ebx, _checkstate_cpustate(,%eax,4)
movl _curproc, %ebx
movl %ebx, _checkstate_curproc(,%eax,4)
movl 12(%esp), %ebx
movl %ebx, _checkstate_pc(,%eax,4)
lock /* checkstate_probed_cpus |= (1<<id) */
btsl %eax, _checkstate_probed_cpus
popl %ds /* restore previous data segment */
popl %ebx
popl %eax
iret
#endif /* BETTER_CLOCK */
/*
* Executed by a CPU when it receives an Xcpuast IPI from another CPU,
*
* - Signals its receipt by clearing bit cpuid in checkstate_need_ast.
*
* - We need a better method of triggering asts on other cpus.
*/
.text
SUPERALIGN_TEXT
.globl _Xcpuast
_Xcpuast:
PUSH_FRAME
movl $KDSEL, %eax
movl %ax, %ds /* use KERNEL data segment */
movl %ax, %es
movl _cpuid, %eax
lock /* checkstate_need_ast &= ~(1<<id) */
btrl %eax, _checkstate_need_ast
movl $0, lapic_eoi /* End Of Interrupt to APIC */
lock
btsl %eax, _checkstate_pending_ast
jc 1f
FAKE_MCOUNT(12*4(%esp))
/*
* Giant locks do not come cheap.
* A lot of cycles are going to be wasted here.
*/
call _get_isrlock
AVCPL_LOCK
#ifdef CPL_AND_CML
movl _cml, %eax
#else
movl _cpl, %eax
#endif
pushl %eax
orl $SWI_AST_PENDING, _ipending
AVCPL_UNLOCK
lock
incb _intr_nesting_level
sti
pushl $0
movl _cpuid, %eax
lock
btrl %eax, _checkstate_pending_ast
lock
btrl %eax, CNAME(resched_cpus)
jz 2f
movl $1, CNAME(want_resched)
lock
incl CNAME(want_resched_cnt)
2:
lock
incl CNAME(cpuast_cnt)
MEXITCOUNT
jmp _doreti
1:
/* We are already in the process of delivering an ast for this CPU */
POP_FRAME
iret
/*
* Executed by a CPU when it receives an XFORWARD_IRQ IPI.
*/
.text
SUPERALIGN_TEXT
.globl _Xforward_irq
_Xforward_irq:
PUSH_FRAME
movl $KDSEL, %eax
movl %ax, %ds /* use KERNEL data segment */
movl %ax, %es
movl $0, lapic_eoi /* End Of Interrupt to APIC */
FAKE_MCOUNT(12*4(%esp))
ISR_TRYLOCK
testl %eax,%eax /* Did we get the lock ? */
jz 1f /* No */
lock
incl CNAME(forward_irq_hitcnt)
cmpb $4, _intr_nesting_level
jae 2f
AVCPL_LOCK
#ifdef CPL_AND_CML
movl _cml, %eax
#else
movl _cpl, %eax
#endif
pushl %eax
AVCPL_UNLOCK
lock
incb _intr_nesting_level
sti
pushl $0
MEXITCOUNT
jmp _doreti /* Handle forwarded interrupt */
1:
lock
incl CNAME(forward_irq_misscnt)
call forward_irq /* Oops, we've lost the isr lock */
MEXITCOUNT
POP_FRAME
iret
2:
lock
incl CNAME(forward_irq_toodeepcnt)
3:
ISR_RELLOCK
MEXITCOUNT
POP_FRAME
iret
/*
*
*/
forward_irq:
MCOUNT
cmpl $0,_invltlb_ok
jz 4f
cmpl $0, CNAME(forward_irq_enabled)
jz 4f
movl _mp_lock,%eax
cmpl $FREE_LOCK,%eax
jne 1f
movl $0, %eax /* Pick CPU #0 if noone has lock */
1:
shrl $24,%eax
movl _cpu_num_to_apic_id(,%eax,4),%ecx
shll $24,%ecx
movl lapic_icr_hi, %eax
andl $~APIC_ID_MASK, %eax
orl %ecx, %eax
movl %eax, lapic_icr_hi
2:
movl lapic_icr_lo, %eax
andl $APIC_DELSTAT_MASK,%eax
jnz 2b
movl lapic_icr_lo, %eax
andl $APIC_RESV2_MASK, %eax
orl $(APIC_DEST_DESTFLD|APIC_DELMODE_FIXED|XFORWARD_IRQ_OFFSET), %eax
movl %eax, lapic_icr_lo
3:
movl lapic_icr_lo, %eax
andl $APIC_DELSTAT_MASK,%eax
jnz 3b
4:
ret
/*
* Executed by a CPU when it receives an Xcpustop IPI from another CPU,
*
* - Signals its receipt.
* - Waits for permission to restart.
* - Signals its restart.
*/
.text
SUPERALIGN_TEXT
.globl _Xcpustop
_Xcpustop:
pushl %ebp
movl %esp, %ebp
pushl %eax
pushl %ecx
pushl %edx
pushl %ds /* save current data segment */
pushl %es
movl $KDSEL, %eax
movl %ax, %ds /* use KERNEL data segment */
movl $0, lapic_eoi /* End Of Interrupt to APIC */
movl _cpuid, %eax
imull $PCB_SIZE, %eax
leal CNAME(stoppcbs)(%eax), %eax
pushl %eax
call CNAME(savectx) /* Save process context */
addl $4, %esp
movl _cpuid, %eax
lock
btsl %eax, _stopped_cpus /* stopped_cpus |= (1<<id) */
1:
btl %eax, _started_cpus /* while (!(started_cpus & (1<<id))) */
jnc 1b
lock
btrl %eax, _started_cpus /* started_cpus &= ~(1<<id) */
lock
btrl %eax, _stopped_cpus /* stopped_cpus &= ~(1<<id) */
test %eax, %eax
jnz 2f
movl CNAME(cpustop_restartfunc), %eax
test %eax, %eax
jz 2f
movl $0, CNAME(cpustop_restartfunc) /* One-shot */
call %eax
2:
popl %es
popl %ds /* restore previous data segment */
popl %edx
popl %ecx
popl %eax
movl %ebp, %esp
popl %ebp
iret
MCOUNT_LABEL(bintr)
FAST_INTR(0,fastintr0)
FAST_INTR(1,fastintr1)
FAST_INTR(2,fastintr2)
FAST_INTR(3,fastintr3)
FAST_INTR(4,fastintr4)
FAST_INTR(5,fastintr5)
FAST_INTR(6,fastintr6)
FAST_INTR(7,fastintr7)
FAST_INTR(8,fastintr8)
FAST_INTR(9,fastintr9)
FAST_INTR(10,fastintr10)
FAST_INTR(11,fastintr11)
FAST_INTR(12,fastintr12)
FAST_INTR(13,fastintr13)
FAST_INTR(14,fastintr14)
FAST_INTR(15,fastintr15)
FAST_INTR(16,fastintr16)
FAST_INTR(17,fastintr17)
FAST_INTR(18,fastintr18)
FAST_INTR(19,fastintr19)
FAST_INTR(20,fastintr20)
FAST_INTR(21,fastintr21)
FAST_INTR(22,fastintr22)
FAST_INTR(23,fastintr23)
INTR(0,intr0)
INTR(1,intr1)
INTR(2,intr2)
INTR(3,intr3)
INTR(4,intr4)
INTR(5,intr5)
INTR(6,intr6)
INTR(7,intr7)
INTR(8,intr8)
INTR(9,intr9)
INTR(10,intr10)
INTR(11,intr11)
INTR(12,intr12)
INTR(13,intr13)
INTR(14,intr14)
INTR(15,intr15)
INTR(16,intr16)
INTR(17,intr17)
INTR(18,intr18)
INTR(19,intr19)
INTR(20,intr20)
INTR(21,intr21)
INTR(22,intr22)
INTR(23,intr23)
MCOUNT_LABEL(eintr)
.data
/*
* Addresses of interrupt handlers.
* XresumeNN: Resumption addresses for HWIs.
*/
.globl _ihandlers
_ihandlers:
/*
* used by:
* ipl.s: doreti_unpend
*/
.long Xresume0, Xresume1, Xresume2, Xresume3
.long Xresume4, Xresume5, Xresume6, Xresume7
.long Xresume8, Xresume9, Xresume10, Xresume11
.long Xresume12, Xresume13, Xresume14, Xresume15
.long Xresume16, Xresume17, Xresume18, Xresume19
.long Xresume20, Xresume21, Xresume22, Xresume23
/*
* used by:
* ipl.s: doreti_unpend
* apic_ipl.s: splz_unpend
*/
.long _swi_null, swi_net, _swi_null, _swi_null
.long _swi_vm, _swi_null, _softclock, swi_ast
imasks: /* masks for interrupt handlers */
.space NHWI*4 /* padding; HWI masks are elsewhere */
.long SWI_TTY_MASK, SWI_NET_MASK, SWI_CAMNET_MASK, SWI_CAMBIO_MASK
.long SWI_VM_MASK, 0, SWI_CLOCK_MASK, SWI_AST_MASK
/* active flag for lazy masking */
iactive:
.long 0
#ifdef COUNT_XINVLTLB_HITS
.globl _xhits
_xhits:
.space (NCPU * 4), 0
#endif /* COUNT_XINVLTLB_HITS */
/* variables used by stop_cpus()/restart_cpus()/Xcpustop */
.globl _stopped_cpus, _started_cpus
_stopped_cpus:
.long 0
_started_cpus:
.long 0
#ifdef BETTER_CLOCK
.globl _checkstate_probed_cpus
_checkstate_probed_cpus:
.long 0
#endif /* BETTER_CLOCK */
.globl _checkstate_need_ast
_checkstate_need_ast:
.long 0
_checkstate_pending_ast:
.long 0
.globl CNAME(forward_irq_misscnt)
.globl CNAME(forward_irq_toodeepcnt)
.globl CNAME(forward_irq_hitcnt)
.globl CNAME(resched_cpus)
.globl CNAME(want_resched_cnt)
.globl CNAME(cpuast_cnt)
.globl CNAME(cpustop_restartfunc)
CNAME(forward_irq_misscnt):
.long 0
CNAME(forward_irq_hitcnt):
.long 0
CNAME(forward_irq_toodeepcnt):
.long 0
CNAME(resched_cpus):
.long 0
CNAME(want_resched_cnt):
.long 0
CNAME(cpuast_cnt):
.long 0
CNAME(cpustop_restartfunc):
.long 0
.globl _apic_pin_trigger
_apic_pin_trigger:
.long 0
/*
* Interrupt counters and names. The format of these and the label names
* must agree with what vmstat expects. The tables are indexed by device
* ids so that we don't have to move the names around as devices are
* attached.
*/
#include "vector.h"
.globl _intrcnt, _eintrcnt
_intrcnt:
.space (NR_DEVICES + ICU_LEN) * 4
_eintrcnt:
.globl _intrnames, _eintrnames
_intrnames:
.ascii DEVICE_NAMES
.asciz "stray irq0"
.asciz "stray irq1"
.asciz "stray irq2"
.asciz "stray irq3"
.asciz "stray irq4"
.asciz "stray irq5"
.asciz "stray irq6"
.asciz "stray irq7"
.asciz "stray irq8"
.asciz "stray irq9"
.asciz "stray irq10"
.asciz "stray irq11"
.asciz "stray irq12"
.asciz "stray irq13"
.asciz "stray irq14"
.asciz "stray irq15"
.asciz "stray irq16"
.asciz "stray irq17"
.asciz "stray irq18"
.asciz "stray irq19"
.asciz "stray irq20"
.asciz "stray irq21"
.asciz "stray irq22"
.asciz "stray irq23"
_eintrnames:
.text
--0-2067406009-930333724=:3553
Content-Type: *UNKNOWN*/X-SH
Content-ID: <Pine.BSF.3.95.990625110013.3553H@current1.whistle.com>
Content-Description: /sys/i386/isa/apic_ipl.s
/*-
* Copyright (c) 1997, by Steve Passe
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. The name of the developer may NOT be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* $Id: apic_ipl.s,v 1.22 1998/09/06 22:41:41 tegge Exp $
*/
.data
ALIGN_DATA
/* current INTerrupt level */
.globl _cil
_cil: .long 0
/* current INTerrupt level mask */
.globl _cml
_cml: .long 0
/*
* Routines used by splz_unpend to build an interrupt frame from a
* trap frame. The _vec[] routines build the proper frame on the stack,
* then call one of _Xintr0 thru _XintrNN.
*
* used by:
* i386/isa/apic_ipl.s (this file): splz_unpend JUMPs to HWIs.
* i386/isa/clock.c: setup _vec[clock] to point at _vec8254.
*/
.globl _vec
_vec:
.long vec0, vec1, vec2, vec3, vec4, vec5, vec6, vec7
.long vec8, vec9, vec10, vec11, vec12, vec13, vec14, vec15
.long vec16, vec17, vec18, vec19, vec20, vec21, vec22, vec23
/*
* Note:
* This is the UP equivilant of _imen.
* It is OPAQUE, and must NOT be accessed directly.
* It MUST be accessed along with the IO APIC as a 'critical region'.
* Accessed by:
* INTREN()
* INTRDIS()
* MAYBE_MASK_IRQ
* MAYBE_UNMASK_IRQ
* imen_dump()
*/
.align 2 /* MUST be 32bit aligned */
.globl _apic_imen
_apic_imen:
.long HWI_MASK
/*
*
*/
.text
SUPERALIGN_TEXT
/*
* Interrupt priority mechanism
* -- soft splXX masks with group mechanism (cpl)
* -- h/w masks for currently active or unused interrupts (imen)
* -- ipending = active interrupts currently masked by cpl
*/
ENTRY(splz)
/*
* The caller has restored cpl and checked that (ipending & ~cpl)
* is nonzero. We have to repeat the check since if there is an
* interrupt while we're looking, _doreti processing for the
* interrupt will handle all the unmasked pending interrupts
* because we restored early. We're repeating the calculation
* of (ipending & ~cpl) anyway so that the caller doesn't have
* to pass it, so this only costs one "jne". "bsfl %ecx,%ecx"
* is undefined when %ecx is 0 so we can't rely on the secondary
* btrl tests.
*/
AICPL_LOCK
movl _cpl,%eax
#ifdef CPL_AND_CML
orl _cml, %eax /* add cml to cpl */
#endif
splz_next:
/*
* We don't need any locking here. (ipending & ~cpl) cannot grow
* while we're looking at it - any interrupt will shrink it to 0.
*/
movl %eax,%ecx
notl %ecx /* set bit = unmasked level */
andl _ipending,%ecx /* set bit = unmasked pending INT */
jne splz_unpend
AICPL_UNLOCK
ret
ALIGN_TEXT
splz_unpend:
bsfl %ecx,%ecx
btrl %ecx,_ipending
jnc splz_next
cmpl $NHWI,%ecx
jae splz_swi
/*
* We would prefer to call the intr handler directly here but that
* doesn't work for badly behaved handlers that want the interrupt
* frame. Also, there's a problem determining the unit number.
* We should change the interface so that the unit number is not
* determined at config time.
*
* The vec[] routines build the proper frame on the stack,
* then call one of _Xintr0 thru _XintrNN.
*/
pushl %ecx
AICPL_UNLOCK
popl %ecx
jmp *_vec(,%ecx,4)
ALIGN_TEXT
splz_swi:
cmpl $SWI_AST,%ecx
je splz_next /* "can't happen" */
pushl %eax
orl imasks(,%ecx,4),%eax
movl %eax,_cpl
pushl %ecx
AICPL_UNLOCK
popl %ecx
call *_ihandlers(,%ecx,4)
AICPL_LOCK
popl %eax
movl %eax,_cpl
jmp splz_next
/*
* Fake clock interrupt(s) so that they appear to come from our caller instead
* of from here, so that system profiling works.
* XXX do this more generally (for all vectors; look up the C entry point).
* XXX frame bogusness stops us from just jumping to the C entry point.
* We have to clear iactive since this is an unpend call, and it will be
* set from the time of the original INT.
*/
/*
* The 'generic' vector stubs.
*/
#define BUILD_VEC(irq_num) \
ALIGN_TEXT ; \
__CONCAT(vec,irq_num): ; \
popl %eax ; \
pushfl ; \
pushl $KCSEL ; \
pushl %eax ; \
cli ; \
lock ; /* MP-safe */ \
andl $~IRQ_BIT(irq_num), iactive ; /* lazy masking */ \
MEXITCOUNT ; \
APIC_ITRACE(apic_itrace_splz, irq_num, APIC_ITRACE_SPLZ) ; \
jmp __CONCAT(_Xintr,irq_num)
BUILD_VEC(0)
BUILD_VEC(1)
BUILD_VEC(2)
BUILD_VEC(3)
BUILD_VEC(4)
BUILD_VEC(5)
BUILD_VEC(6)
BUILD_VEC(7)
BUILD_VEC(8)
BUILD_VEC(9)
BUILD_VEC(10)
BUILD_VEC(11)
BUILD_VEC(12)
BUILD_VEC(13)
BUILD_VEC(14)
BUILD_VEC(15)
BUILD_VEC(16) /* 8 additional INTs in IO APIC */
BUILD_VEC(17)
BUILD_VEC(18)
BUILD_VEC(19)
BUILD_VEC(20)
BUILD_VEC(21)
BUILD_VEC(22)
BUILD_VEC(23)
/******************************************************************************
* XXX FIXME: figure out where these belong.
*/
/* this nonsense is to verify that masks ALWAYS have 1 and only 1 bit set */
#define QUALIFY_MASKS_NOT
#ifdef QUALIFY_MASKS
#define QUALIFY_MASK \
btrl %ecx, %eax ; \
andl %eax, %eax ; \
jz 1f ; \
pushl $bad_mask ; \
call _panic ; \
1:
bad_mask: .asciz "bad mask"
#else
#define QUALIFY_MASK
#endif
/*
* (soon to be) MP-safe function to clear ONE INT mask bit.
* The passed arg is a 32bit u_int MASK.
* It sets the associated bit in _apic_imen.
* It sets the mask bit of the associated IO APIC register.
*/
ENTRY(INTREN)
pushfl /* save state of EI flag */
cli /* prevent recursion */
IMASK_LOCK /* enter critical reg */
movl 8(%esp), %eax /* mask into %eax */
bsfl %eax, %ecx /* get pin index */
btrl %ecx, _apic_imen /* update _apic_imen */
QUALIFY_MASK
shll $4, %ecx
movl CNAME(int_to_apicintpin) + 8(%ecx), %edx
movl CNAME(int_to_apicintpin) + 12(%ecx), %ecx
movl %ecx, (%edx) /* write the target register index */
movl 16(%edx), %eax /* read the target register data */
andl $~IOART_INTMASK, %eax /* clear mask bit */
movl %eax, 16(%edx) /* write the APIC register data */
IMASK_UNLOCK /* exit critical reg */
popfl /* restore old state of EI flag */
ret
/*
* (soon to be) MP-safe function to set ONE INT mask bit.
* The passed arg is a 32bit u_int MASK.
* It clears the associated bit in _apic_imen.
* It clears the mask bit of the associated IO APIC register.
*/
ENTRY(INTRDIS)
pushfl /* save state of EI flag */
cli /* prevent recursion */
IMASK_LOCK /* enter critical reg */
movl 8(%esp), %eax /* mask into %eax */
bsfl %eax, %ecx /* get pin index */
btsl %ecx, _apic_imen /* update _apic_imen */
QUALIFY_MASK
shll $4, %ecx
movl CNAME(int_to_apicintpin) + 8(%ecx), %edx
movl CNAME(int_to_apicintpin) + 12(%ecx), %ecx
movl %ecx, (%edx) /* write the target register index */
movl 16(%edx), %eax /* read the target register data */
orl $IOART_INTMASK, %eax /* set mask bit */
movl %eax, 16(%edx) /* write the APIC register data */
IMASK_UNLOCK /* exit critical reg */
popfl /* restore old state of EI flag */
ret
/******************************************************************************
*
*/
/*
* void write_ioapic_mask(int apic, u_int mask);
*/
#define _INT_MASK 0x00010000
#define _PIN_MASK 0x00ffffff
#define _OLD_ESI 0(%esp)
#define _OLD_EBX 4(%esp)
#define _RETADDR 8(%esp)
#define _APIC 12(%esp)
#define _MASK 16(%esp)
.align 2
write_ioapic_mask:
pushl %ebx /* scratch */
pushl %esi /* scratch */
movl _apic_imen, %ebx
xorl _MASK, %ebx /* %ebx = _apic_imen ^ mask */
andl $_PIN_MASK, %ebx /* %ebx = _apic_imen & 0x00ffffff */
jz all_done /* no change, return */
movl _APIC, %esi /* APIC # */
movl _ioapic(,%esi,4), %esi /* %esi holds APIC base address */
next_loop: /* %ebx = diffs, %esi = APIC base */
bsfl %ebx, %ecx /* %ecx = index if 1st/next set bit */
jz all_done
btrl %ecx, %ebx /* clear this bit in diffs */
leal 16(,%ecx,2), %edx /* calculate register index */
movl %edx, (%esi) /* write the target register index */
movl 16(%esi), %eax /* read the target register data */
btl %ecx, _MASK /* test for mask or unmask */
jnc clear /* bit is clear */
orl $_INT_MASK, %eax /* set mask bit */
jmp write
clear: andl $~_INT_MASK, %eax /* clear mask bit */
write: movl %eax, 16(%esi) /* write the APIC register data */
jmp next_loop /* try another pass */
all_done:
popl %esi
popl %ebx
ret
#undef _OLD_ESI
#undef _OLD_EBX
#undef _RETADDR
#undef _APIC
#undef _MASK
#undef _PIN_MASK
#undef _INT_MASK
#ifdef oldcode
_INTREN:
movl _apic_imen, %eax
notl %eax /* mask = ~mask */
andl _apic_imen, %eax /* %eax = _apic_imen & ~mask */
pushl %eax /* new (future) _apic_imen value */
pushl $0 /* APIC# arg */
call write_ioapic_mask /* modify the APIC registers */
addl $4, %esp /* remove APIC# arg from stack */
popl _apic_imen /* _apic_imen |= mask */
ret
_INTRDIS:
movl _apic_imen, %eax
orl 4(%esp), %eax /* %eax = _apic_imen | mask */
pushl %eax /* new (future) _apic_imen value */
pushl $0 /* APIC# arg */
call write_ioapic_mask /* modify the APIC registers */
addl $4, %esp /* remove APIC# arg from stack */
popl _apic_imen /* _apic_imen |= mask */
ret
#endif /* oldcode */
#ifdef ready
/*
* u_int read_io_apic_mask(int apic);
*/
ALIGN_TEXT
read_io_apic_mask:
ret
/*
* Set INT mask bit for each bit set in 'mask'.
* Ignore INT mask bit for all others.
*
* void set_io_apic_mask(apic, u_int32_t bits);
*/
ALIGN_TEXT
set_io_apic_mask:
ret
/*
* void set_ioapic_maskbit(int apic, int bit);
*/
ALIGN_TEXT
set_ioapic_maskbit:
ret
/*
* Clear INT mask bit for each bit set in 'mask'.
* Ignore INT mask bit for all others.
*
* void clr_io_apic_mask(int apic, u_int32_t bits);
*/
ALIGN_TEXT
clr_io_apic_mask:
ret
/*
* void clr_ioapic_maskbit(int apic, int bit);
*/
ALIGN_TEXT
clr_ioapic_maskbit:
ret
#endif /** ready */
/******************************************************************************
*
*/
/*
* u_int io_apic_write(int apic, int select);
*/
ENTRY(io_apic_read)
movl 4(%esp), %ecx /* APIC # */
movl _ioapic(,%ecx,4), %edx /* APIC base register address */
movl 8(%esp), %eax /* target register index */
movl %eax, (%edx) /* write the target register index */
movl 16(%edx), %eax /* read the APIC register data */
ret /* %eax = register value */
/*
* void io_apic_write(int apic, int select, int value);
*/
ENTRY(io_apic_write)
movl 4(%esp), %ecx /* APIC # */
movl _ioapic(,%ecx,4), %edx /* APIC base register address */
movl 8(%esp), %eax /* target register index */
movl %eax, (%edx) /* write the target register index */
movl 12(%esp), %eax /* target register value */
movl %eax, 16(%edx) /* write the APIC register data */
ret /* %eax = void */
/*
* Send an EOI to the local APIC.
*/
ENTRY(apic_eoi)
movl $0, _lapic+0xb0
ret
--0-2067406009-930333724=:3553
Content-Type: *UNKNOWN*/X-CSH
Content-ID: <Pine.BSF.3.95.990625110013.3553I@current1.whistle.com>
Content-Description: /sys/i386/isa/ipl_funcs.c
/*-
* Copyright (c) 1997 Bruce Evans.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* $Id: ipl_funcs.c,v 1.13 1998/02/01 22:04:58 bde Exp $
*/
#include <sys/types.h>
#include <sys/systm.h>
#include <machine/ipl.h>
#ifndef SMP
#if 0
/*
* The volatile bitmap variables must be set atomically. This normally
* involves using a machine-dependent bit-set or `or' instruction.
*/
#define DO_SETBITS(name, var, bits) \
void name(void) \
{ \
setbits(var, bits); \
}
DO_SETBITS(setdelayed, &ipending, loadandclear((unsigned *)&idelayed))
DO_SETBITS(setsoftast, &ipending, SWI_AST_PENDING)
DO_SETBITS(setsoftcamnet,&ipending, SWI_CAMNET_PENDING)
DO_SETBITS(setsoftcambio,&ipending, SWI_CAMBIO_PENDING)
DO_SETBITS(setsoftclock, &ipending, SWI_CLOCK_PENDING)
DO_SETBITS(setsoftnet, &ipending, SWI_NET_PENDING)
DO_SETBITS(setsofttty, &ipending, SWI_TTY_PENDING)
DO_SETBITS(setsoftvm, &ipending, SWI_VM_PENDING)
DO_SETBITS(schedsoftcamnet, &idelayed, SWI_CAMNET_PENDING)
DO_SETBITS(schedsoftcambio, &idelayed, SWI_CAMBIO_PENDING)
DO_SETBITS(schedsoftnet, &idelayed, SWI_NET_PENDING)
DO_SETBITS(schedsofttty, &idelayed, SWI_TTY_PENDING)
DO_SETBITS(schedsoftvm, &idelayed, SWI_VM_PENDING)
unsigned
softclockpending(void)
{
return (ipending & SWI_CLOCK_PENDING);
}
#define GENSPL(name, set_cpl) \
unsigned name(void) \
{ \
unsigned x; \
\
x = cpl; \
set_cpl; \
return (x); \
}
GENSPL(splbio, cpl |= bio_imask)
GENSPL(splcam, cpl |= cam_imask)
GENSPL(splclock, cpl = HWI_MASK | SWI_MASK)
GENSPL(splhigh, cpl = HWI_MASK | SWI_MASK)
GENSPL(splimp, cpl |= net_imask)
GENSPL(splnet, cpl |= SWI_NET_MASK)
GENSPL(splsoftcam, cpl |= SWI_CAMBIO_MASK | SWI_CAMNET_MASK)
GENSPL(splsoftcambio, cpl |= SWI_CAMBIO_MASK)
GENSPL(splsoftcamnet, cpl |= SWI_CAMNET_MASK)
GENSPL(splsoftclock, cpl = SWI_CLOCK_MASK)
GENSPL(splsofttty, cpl |= SWI_TTY_MASK)
GENSPL(splsoftvm, cpl |= SWI_VM_MASK)
GENSPL(splstatclock, cpl |= stat_imask)
GENSPL(spltty, cpl |= tty_imask)
GENSPL(splvm, cpl |= net_imask | bio_imask)
#endif
void
spl0(void)
{
cpl = SWI_AST_MASK;
if (ipending & ~SWI_AST_MASK)
splz();
}
void
splx(unsigned ipl)
{
cpl = ipl;
if (ipending & ~ipl)
splz();
}
#else /* !SMP */
#include <machine/smp.h>
#include <machine/smptests.h>
#ifndef SPL_DEBUG_POSTCODE
#undef POSTCODE
#undef POSTCODE_LO
#undef POSTCODE_HI
#define POSTCODE(X)
#define POSTCODE_LO(X)
#define POSTCODE_HI(X)
#endif /* SPL_DEBUG_POSTCODE */
/*
* The volatile bitmap variables must be set atomically. This normally
* involves using a machine-dependent bit-set or `or' instruction.
*/
#define DO_SETBITS(name, var, bits) \
void name(void) \
{ \
IFCPL_LOCK(); \
setbits(var, bits); \
IFCPL_UNLOCK(); \
}
DO_SETBITS(setdelayed, &ipending, loadandclear((unsigned *)&idelayed))
DO_SETBITS(setsoftast, &ipending, SWI_AST_PENDING)
DO_SETBITS(setsoftcamnet,&ipending, SWI_CAMNET_PENDING)
DO_SETBITS(setsoftcambio,&ipending, SWI_CAMBIO_PENDING)
DO_SETBITS(setsoftclock, &ipending, SWI_CLOCK_PENDING)
DO_SETBITS(setsoftnet, &ipending, SWI_NET_PENDING)
DO_SETBITS(setsofttty, &ipending, SWI_TTY_PENDING)
DO_SETBITS(setsoftvm, &ipending, SWI_VM_PENDING)
DO_SETBITS(schedsoftcamnet, &idelayed, SWI_CAMNET_PENDING)
DO_SETBITS(schedsoftcambio, &idelayed, SWI_CAMBIO_PENDING)
DO_SETBITS(schedsoftnet, &idelayed, SWI_NET_PENDING)
DO_SETBITS(schedsofttty, &idelayed, SWI_TTY_PENDING)
DO_SETBITS(schedsoftvm, &idelayed, SWI_VM_PENDING)
unsigned
softclockpending(void)
{
unsigned x;
IFCPL_LOCK();
x = ipending & SWI_CLOCK_PENDING;
IFCPL_UNLOCK();
return (x);
}
/*
* This version has to check for bsp_apic_ready,
* as calling simple_lock() (ie ss_lock) before then deadlocks the system.
* A sample count of GENSPL calls before bsp_apic_ready was set: 2193
*/
#ifdef INTR_SPL
#ifdef SPL_DEBUG
#define MAXZ 100000000
#define SPIN_VAR unsigned z;
#define SPIN_RESET z = 0;
#if 0
#define SPIN_SPL \
if (++z >= MAXZ) { \
/* XXX allow lock-free panic */ \
bsp_apic_ready = 0; \
panic("\ncil: 0x%08x", cil); \
}
#else
#define SPIN_SPL \
if (++z >= MAXZ) { \
/* XXX allow lock-free panic */ \
bsp_apic_ready = 0; \
printf("\ncil: 0x%08x", cil); \
breakpoint(); \
}
#endif /* 0/1 */
#else /* SPL_DEBUG */
#define SPIN_VAR
#define SPIN_RESET
#define SPIN_SPL
#endif /* SPL_DEBUG */
#endif
#ifdef INTR_SPL
#define GENSPL(NAME, OP, MODIFIER, PC) \
unsigned NAME(void) \
{ \
unsigned x, y; \
SPIN_VAR; \
\
if (!bsp_apic_ready) { \
x = cpl; \
cpl OP MODIFIER; \
return (x); \
} \
\
for (;;) { \
IFCPL_LOCK(); /* MP-safe */ \
x = y = cpl; /* current value */ \
POSTCODE(0x20 | PC); \
if (inside_intr) \
break; /* XXX only 1 INT allowed */ \
y OP MODIFIER; /* desired value */ \
if (cil & y) { /* not now */ \
IFCPL_UNLOCK(); /* allow cil to change */ \
SPIN_RESET; \
while (cil & y) \
SPIN_SPL \
continue; /* try again */ \
} \
break; \
} \
cpl OP MODIFIER; /* make the change */ \
IFCPL_UNLOCK(); \
\
return (x); \
}
/* NAME: OP: MODIFIER: PC: */
GENSPL(splbio, |=, bio_imask, 2)
GENSPL(splcam, |=, cam_imask, 7)
GENSPL(splclock, =, HWI_MASK | SWI_MASK, 3)
GENSPL(splhigh, =, HWI_MASK | SWI_MASK, 4)
GENSPL(splimp, |=, net_imask, 5)
GENSPL(splnet, |=, SWI_NET_MASK, 6)
GENSPL(splsoftcam, |=, SWI_CAMBIO_MASK | SWI_CAMNET_MASK, 8)
GENSPL(splsoftcambio, |=, SWI_CAMBIO_MASK, 9)
GENSPL(splsoftcamnet, |=, SWI_CAMNET_MASK, 10)
GENSPL(splsoftclock, =, SWI_CLOCK_MASK, 11)
GENSPL(splsofttty, |=, SWI_TTY_MASK, 12)
GENSPL(splsoftvm, |=, SWI_VM_MASK, 16)
GENSPL(splstatclock, |=, stat_imask, 13)
GENSPL(spltty, |=, tty_imask, 14)
GENSPL(splvm, |=, net_imask | bio_imask, 15)
#else /* INTR_SPL */
#define GENSPL(NAME, set_cpl) \
unsigned NAME(void) \
{ \
unsigned x; \
\
if (!bsp_apic_ready) { \
x = cpl; \
set_cpl; \
} \
else { \
IFCPL_LOCK(); \
x = cpl; \
set_cpl; \
IFCPL_UNLOCK(); \
} \
\
return (x); \
}
GENSPL(splbio, cpl |= bio_imask)
GENSPL(splclock, cpl = HWI_MASK | SWI_MASK)
GENSPL(splhigh, cpl = HWI_MASK | SWI_MASK)
GENSPL(splimp, cpl |= net_imask)
GENSPL(splnet, cpl |= SWI_NET_MASK)
GENSPL(splcam, cpl |= cam_imask)
GENSPL(splsoftcam, cpl |= SWI_CAMBIO_MASK | SWI_CAMNET_MASK)
GENSPL(splsoftcambio, cpl |= SWI_CAMBIO_MASK)
GENSPL(splsoftcamnet, cpl |= SWI_CAMNET_MASK)
GENSPL(splsoftclock, cpl = SWI_CLOCK_MASK)
GENSPL(splsofttty, cpl |= SWI_TTY_MASK)
GENSPL(splsoftvm, cpl |= SWI_VM_MASK)
GENSPL(splstatclock, cpl |= stat_imask)
GENSPL(spltty, cpl |= tty_imask)
GENSPL(splvm, cpl |= net_imask | bio_imask)
#endif /* INTR_SPL */
void
spl0(void)
{
int unpend;
#ifdef INTR_SPL
SPIN_VAR;
for (;;) {
IFCPL_LOCK();
POSTCODE_HI(0xc);
if (cil & SWI_AST_MASK) { /* not now */
IFCPL_UNLOCK(); /* allow cil to change */
SPIN_RESET;
while (cil & SWI_AST_MASK)
SPIN_SPL
continue; /* try again */
}
break;
}
#else /* INTR_SPL */
IFCPL_LOCK();
#endif /* INTR_SPL */
cpl = SWI_AST_MASK;
unpend = ipending & ~SWI_AST_MASK;
IFCPL_UNLOCK();
if (unpend && !inside_intr)
splz();
}
void
splx(unsigned ipl)
{
int unpend;
#ifdef INTR_SPL
SPIN_VAR;
#endif
if (!bsp_apic_ready) {
cpl = ipl;
if (ipending & ~ipl)
splz();
return;
}
#ifdef INTR_SPL
for (;;) {
IFCPL_LOCK();
POSTCODE_HI(0xe);
if (inside_intr)
break; /* XXX only 1 INT allowed */
POSTCODE_HI(0xf);
if (cil & ipl) { /* not now */
IFCPL_UNLOCK(); /* allow cil to change */
SPIN_RESET;
while (cil & ipl)
SPIN_SPL
continue; /* try again */
}
break;
}
#else /* INTR_SPL */
IFCPL_LOCK();
#endif /* INTR_SPL */
cpl = ipl;
unpend = ipending & ~ipl;
IFCPL_UNLOCK();
if (unpend && !inside_intr)
splz();
}
/*
* Replaces UP specific inline found in (?) pci/pci_support.c.
*
* Stefan said:
* You know, that splq() is used in the shared interrupt multiplexer, and that
* the SMP version should not have too much overhead. If it is significantly
* slower, then moving the splq() out of the loop in intr_mux() and passing in
* the logical OR of all mask values might be a better solution than the
* current code. (This logical OR could of course be pre-calculated whenever
* another shared interrupt is registered ...)
*/
intrmask_t
splq(intrmask_t mask)
{
intrmask_t tmp, tmp2;
#ifdef INTR_SPL
for (;;) {
IFCPL_LOCK();
tmp = tmp2 = cpl;
tmp2 |= mask;
if (cil & tmp2) { /* not now */
IFCPL_UNLOCK(); /* allow cil to change */
while (cil & tmp2)
/* spin */ ;
continue; /* try again */
}
break;
}
cpl = tmp2;
#else /* INTR_SPL */
IFCPL_LOCK();
tmp = cpl;
cpl |= mask;
#endif /* INTR_SPL */
IFCPL_UNLOCK();
return (tmp);
}
#endif /* !SMP */
--0-2067406009-930333724=:3553--
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-smp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.3.95.990625110011.3553C-100000>