From owner-freebsd-acpi@FreeBSD.ORG Wed May 21 17:23:39 2008 Return-Path: Delivered-To: freebsd-acpi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A09EF106564A for ; Wed, 21 May 2008 17:23:39 +0000 (UTC) (envelope-from vova@sw.ru) Received: from relay.sw.ru (mailhub.sw.ru [195.214.232.25]) by mx1.freebsd.org (Postfix) with ESMTP id 41E758FC1A for ; Wed, 21 May 2008 17:23:36 +0000 (UTC) (envelope-from vova@sw.ru) Received: from vbook.fbsd.ru ([10.30.1.111]) (authenticated bits=0) by relay.sw.ru (8.13.4/8.13.4) with ESMTP id m4LHNSAK015070 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 21 May 2008 21:23:32 +0400 (MSD) Received: from vova by vbook.fbsd.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Jys2A-0000kR-Co; Wed, 21 May 2008 21:23:26 +0400 From: Vladimir Grebenschikov To: takawata@init-main.com In-Reply-To: <1211379715.2851.2.camel@localhost> References: <200805131125.m4DBPu1q092741@sana.init-main.com> <1211379715.2851.2.camel@localhost> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Organization: SWsoft Date: Wed, 21 May 2008 21:23:25 +0400 Message-Id: <1211390605.1794.15.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.22.1 FreeBSD GNOME Team Port Sender: Vladimir Grebenschikov Cc: freebsd-acpi@freebsd.org Subject: Re: SMP suspend/resume. X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vova@fbsd.ru List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2008 17:23:39 -0000 On Wed, 2008-05-21 at 18:21 +0400, Vladimir Grebenschikov wrote: > On Tue, 2008-05-13 at 20:25 +0900, takawata@init-main.com wrote: > > Hi, I managed to make suspend and resume work on SMP system. > > The patch following is a bit crude patch, but it begin=20 > > to work on my ThinkPad X61 (core2duo system). >=20 > Tried patch on T60, from text console,=20 > on acpiconf -s3 > it shows multiple times "forward_wakeup: Idle processor not found" on > console and then friezed hard (even DDB does not works). But nothing was > suspended actually ( before system goes to sleep but did not awakes) >=20 > Any hints ? In some conditions (single-user, no additional drivers) it shows: forward_wakeup: Idle processor not found PREP_WALK LEAVE_SLEEP_STATE DEVICE_RESUME =EF=BB=BFforward_wakeup: Idle processor not found ENABLE_FIXED_EVENT =EF=BB=BFforward_wakeup: Idle processor not found =EF=BB=BFforward_wakeup: Idle processor not found =EF=BB=BFforward_wakeup: Idle processor not found =EF=BB=BFforward_wakeup: Idle processor not found =EF=BB=BFforward_wakeup: Idle processor not found ... etc ... sometimes it shows just many =EF=BB=BFforward_wakeup: Idle processor not found and in any case locks hard and did not enter S3 But RELENG_7 (without patch) normally enters to S3, but failed to wake. > > TODO: > > 1. Suspend/resume path it self is simular to AP boot path. > > Some of code may be integrated. > > 2. More context, like MTRR or npx context should be saved on=20 > > suspend. > > 3. Make acpi suspend resume path more ABI aware: needless=20 > > register recoverly or special register context saving=20 > > (the value itself is usually constant) should be removed. > > 4. Make same binary module work on both UP or SMP case. > > (Or is it time to give up using acpi module on also on i386?) > >=20 > >=20 > >=20 > >=20 > >=20 > > Index: i386/acpica/acpi_wakeup.c > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > RCS file: /home/ncvs/src/sys/i386/acpica/acpi_wakeup.c,v > > retrieving revision 1.47 > > diff -u -r1.47 acpi_wakeup.c > > --- i386/acpica/acpi_wakeup.c 16 Mar 2008 10:58:03 -0000 1.47 > > +++ i386/acpica/acpi_wakeup.c 13 May 2008 09:12:18 -0000 > > @@ -27,6 +27,7 @@ > > =20 > > #include > > __FBSDID("$FreeBSD: src/sys/i386/acpica/acpi_wakeup.c,v 1.47 2008/03/1= 6 10:58:03 rwatson Exp $"); > > +#define SMP > > =20 > > #include > > #include > > @@ -49,6 +50,11 @@ > > =20 > > #include > > #include > > +#include > > +#include > > +#include > > +#include > > +#include > > =20 > > #include "acpi_wakecode.h" > > =20 > > @@ -71,7 +77,9 @@ > > =20 > > static uint16_t r_cs, r_ds, r_es, r_fs, r_gs, r_ss, r_tr; > > static uint32_t r_esp; > > - > > +extern void *bootstacks[]; > > +static char *bootSTK; > > +void restore_sub(void); > > static void acpi_printcpu(void); > > static void acpi_realmodeinst(void *arg, bus_dma_segment_t *segs, > > int nsegs, int error); > > @@ -80,6 +88,7 @@ > > /* XXX shut gcc up */ > > extern int acpi_savecpu(void); > > extern int acpi_restorecpu(void); > > +extern void acpi_kicksub(void); > > =20 > > #ifdef __GNUCLIKE_ASM > > __asm__(" \n\ > > @@ -104,6 +113,15 @@ > > movl %eax,(%esp) \n\ > > xorl %eax,%eax \n\ > > ret \n\ > > + \n\ > > + .text \n\ > > + .p2align 2, 0x90 \n\ > > + .type acpi_kicksub, @function \n\ > > +acpi_kicksub: \n\ > > + .align 4 \n\ > > + movl bootSTK,%esp \n\ > > + jmp restore_sub \n\ > > + ret \n\ > > \n\ > > .text \n\ > > .p2align 2, 0x90 \n\ > > @@ -149,6 +167,24 @@ > > ret \n\ > > "); > > #endif /* __GNUCLIKE_ASM */ > > +int acpi_cpu_resumed[MAXCPU]; > > +int acpi_curcpu; > > +extern int switch_debug; > > + > > +void restore_sub() > > +{ > > + ACPI_DISABLE_IRQS(); > > + printf("RESTORE_SUB\n"); > > + lapic_disable();=09 > > + printf("LAPIC_SETUP\n"); > > + lapic_setup(0);=09 > > + lapic_dump("RESTORE_SUB"); > > + printf("RESTORE_SUB2\n"); > > + ACPI_ENABLE_IRQS(); > > + > > + acpi_cpu_resumed[acpi_curcpu]=3D 1; > > + acpi_restorecpu(); > > +} > > =20 > > static void > > acpi_printcpu(void) > > @@ -187,6 +223,119 @@ > > outb(0x61, inb(0x61) & ~0x3); > > } > > =20 > > + > > +int resume_other_cpu(struct acpi_softc *sc, int cpu); > > +int resume_other_cpu(struct acpi_softc *sc, int cpu) > > +{ > > + int ms; > > + int apic_id =3D cpu_apic_ids[cpu]; > > + int gsel_tss; > > + > > + gsel_tss =3D GSEL(GPROC0_SEL, SEL_KPL); > > + acpi_curcpu =3D cpu; > > + bootSTK=3D (char *)bootstacks[cpu] + KSTACK_PAGES * PAGE_SIZE - 4; > > + printf("%p\n", bootSTK);=09 > > + p_gdt =3D (struct region_descriptor *) > > + (sc->acpi_wakeaddr + physical_gdt); > > + saved_gdt.rd_limit =3D NGDT * sizeof(gdt[0]) -1; > > + saved_gdt.rd_base =3D (int )&gdt[cpu*NGDT]; > > + p_gdt->rd_limit =3D saved_gdt.rd_limit; > > + p_gdt->rd_base =3D vtophys(saved_gdt.rd_base); > > + r_esp =3D stoppcbs[cpu].pcb_esp; > > + r_ebp =3D stoppcbs[cpu].pcb_ebp; > > + r_esi =3D stoppcbs[cpu].pcb_esi; > > + r_edi =3D stoppcbs[cpu].pcb_edi; > > + r_efl =3D stoppcbs[cpu].pcb_psl; > > + ret_addr =3D stoppcbs[cpu].pcb_eip; > > + WAKECODE_FIXUP(physical_esp, uint32_t, vtophys(bootSTK) ); > > + WAKECODE_FIXUP(previous_cr0, uint32_t, r_cr0); > > + WAKECODE_FIXUP(previous_cr2, uint32_t, r_cr2); > > + WAKECODE_FIXUP(previous_cr3, uint32_t, r_cr3); > > + WAKECODE_FIXUP(previous_cr4, uint32_t, r_cr4); > > +=09 > > + WAKECODE_FIXUP(resume_beep, uint32_t, 0); > > + WAKECODE_FIXUP(reset_video, uint32_t, 0); > > +=09 > > + WAKECODE_FIXUP(previous_tr, uint16_t, gsel_tss); > > + WAKECODE_BCOPY(previous_gdt, struct region_descriptor, saved_gdt); > > + WAKECODE_FIXUP(previous_ldt, uint16_t, saved_ldt); > > + WAKECODE_BCOPY(previous_idt, struct region_descriptor, saved_idt); > > +=09 > > + WAKECODE_FIXUP(where_to_recover, void *, acpi_kicksub); > > +=09 > > + WAKECODE_FIXUP(previous_ds, uint16_t, r_ds); > > + WAKECODE_FIXUP(previous_es, uint16_t, r_es); > > + WAKECODE_FIXUP(previous_fs, uint16_t, r_fs); > > + WAKECODE_FIXUP(previous_gs, uint16_t, 0); > > + WAKECODE_FIXUP(previous_ss, uint16_t, r_ss); > > + > > + /* do an INIT IPI: assert RESET */ > > + lapic_ipi_raw(APIC_DEST_DESTFLD | APIC_TRIGMOD_EDGE | > > + APIC_LEVEL_ASSERT | APIC_DESTMODE_PHY | APIC_DELMODE_INIT, apic_i= d); > > + > > + /* wait for pending status end */ > > + lapic_ipi_wait(-1); > > + > > + /* do an INIT IPI: deassert RESET */ > > + lapic_ipi_raw(APIC_DEST_ALLESELF | APIC_TRIGMOD_LEVEL | > > + APIC_LEVEL_DEASSERT | APIC_DESTMODE_PHY | APIC_DELMODE_INIT, 0); > > + > > + /* wait for pending status end */ > > + DELAY(10000); /* wait ~10mS */ > > + lapic_ipi_wait(-1); > > + /* > > + * next we do a STARTUP IPI: the previous INIT IPI might still be > > + * latched, (P5 bug) this 1st STARTUP would then terminate > > + * immediately, and the previously started INIT IPI would continue. O= R > > + * the previous INIT IPI has already run. and this STARTUP IPI will > > + * run. OR the previous INIT IPI was ignored. and this STARTUP IPI > > + * will run. > > + */ > > + > > + /* do a STARTUP IPI */ > > + lapic_ipi_raw(APIC_DEST_DESTFLD | APIC_TRIGMOD_EDGE | > > + APIC_LEVEL_DEASSERT | APIC_DESTMODE_PHY | APIC_DELMODE_STARTUP | > > + ((sc->acpi_wakephys >>12)&0xff), apic_id); > > + lapic_ipi_wait(-1); > > + DELAY(200); /* wait ~200uS */ > > + > > + /* > > + * finally we do a 2nd STARTUP IPI: this 2nd STARTUP IPI should run I= F > > + * the previous STARTUP IPI was cancelled by a latched INIT IPI. OR > > + * this STARTUP IPI will be ignored, as only ONE STARTUP IPI is > > + * recognized after hardware RESET or INIT IPI. > > + */ > > + > > + lapic_ipi_raw(APIC_DEST_DESTFLD | APIC_TRIGMOD_EDGE | > > + APIC_LEVEL_DEASSERT | APIC_DESTMODE_PHY | APIC_DELMODE_STARTUP | > > + ((sc->acpi_wakephys >>12)&0xff), apic_id); > > + lapic_ipi_wait(-1); > > + DELAY(200); /* wait ~200uS */ > > + > > + /* Wait up to 5 seconds for it to start. */ > > + for (ms =3D 0; ms < 5000; ms++) { > > + if(acpi_cpu_resumed[cpu]){ > > + acpi_cpu_resumed[cpu]=3D 0; > > + return 0; > > + } > > + DELAY(1000); > > + } > > + return -1; /* return FAILURE */ > > + > > +} > > +int resume_other_cpus(struct acpi_softc *sc); > > +int resume_other_cpus(struct acpi_softc *sc) > > +{ > > + int i; > > + printf("RESUME_OTHER_CPUS"); > > + *((volatile u_short *) 0x467) =3D 0; > > + *((volatile u_short *) 0x468) =3D (sc->acpi_wakephys&0xffff0)>>4; > > + > > + for(i =3D 1; i < mp_ncpus; i++){ > > + resume_other_cpu(sc, i); > > + } > > + return 0; > > +} > > int > > acpi_sleep_machdep(struct acpi_softc *sc, int state) > > { > > @@ -270,14 +419,15 @@ > > for (;;) ; > > } else { > > /* Execute Wakeup */ > > - intr_resume(); > > - > > if (bootverbose) { > > acpi_savecpu(); > > acpi_printcpu(); > > } > > + resume_other_cpus(sc); > > + restart_cpus(stopped_cpus); > > + intr_resume(); > > + lapic_dump("MAIN"); > > } > > - > > out: > > load_cr3(cr3); > > write_eflags(ef); > > @@ -285,7 +435,7 @@ > > /* If we beeped, turn it off after a delay. */ > > if (acpi_resume_beep) > > timeout(acpi_stop_beep, NULL, 3 * hz); > > - > > + printf("FUGAFUGA\n"); > > return (ret); > > } > > =20 > > Index: i386/i386/io_apic.c > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > RCS file: /home/ncvs/src/sys/i386/i386/io_apic.c,v > > retrieving revision 1.35 > > diff -u -r1.35 io_apic.c > > --- i386/i386/io_apic.c 5 Jun 2007 18:57:48 -0000 1.35 > > +++ i386/i386/io_apic.c 13 May 2008 08:22:55 -0000 > > @@ -444,8 +444,9 @@ > > struct ioapic *io =3D (struct ioapic *)pic; > > int i; > > =20 > > - for (i =3D 0; i < io->io_numintr; i++) > > + for (i =3D 0; i < io->io_numintr; i++){ > > ioapic_program_intpin(&io->io_pins[i]); > > + } > > } > > =20 > > /* > > Index: i386/i386/mp_machdep.c > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > RCS file: /home/ncvs/src/sys/i386/i386/mp_machdep.c,v > > retrieving revision 1.286 > > diff -u -r1.286 mp_machdep.c > > --- i386/i386/mp_machdep.c 10 Apr 2008 18:38:31 -0000 1.286 > > +++ i386/i386/mp_machdep.c 13 May 2008 07:08:29 -0000 > > @@ -1299,18 +1299,19 @@ > > int cpu =3D PCPU_GET(cpuid); > > int cpumask =3D PCPU_GET(cpumask); > > =20 > > - savectx(&stoppcbs[cpu]); > > - > > - /* Indicate that we are stopped */ > > - atomic_set_int(&stopped_cpus, cpumask); > > + if(savectx(&stoppcbs[cpu])){ > > + /* Indicate that we are stopped */ > > + atomic_set_int(&stopped_cpus, cpumask); > > + wbinvd(); > > + } > > =20 > > /* Wait for restart */ > > - while (!(started_cpus & cpumask)) > > - ia32_pause(); > > - > > + while (!(started_cpus & cpumask)){ > > + ia32_pause(); > > + } > > atomic_clear_int(&started_cpus, cpumask); > > atomic_clear_int(&stopped_cpus, cpumask); > > - > > +=09 > > if (cpu =3D=3D 0 && cpustop_restartfunc !=3D NULL) { > > cpustop_restartfunc(); > > cpustop_restartfunc =3D NULL; > > Index: i386/i386/swtch.s > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > RCS file: /home/ncvs/src/sys/i386/i386/swtch.s,v > > retrieving revision 1.156 > > diff -u -r1.156 swtch.s > > --- i386/i386/swtch.s 22 Aug 2007 05:06:14 -0000 1.156 > > +++ i386/i386/swtch.s 9 May 2008 15:16:03 -0000 > > @@ -413,6 +413,6 @@ > > 1: > > popfl > > #endif /* DEV_NPX */ > > - > > + movl $1, %eax > > ret > > END(savectx) > > Index: i386/include/pcb.h > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > RCS file: /home/ncvs/src/sys/i386/include/pcb.h,v > > retrieving revision 1.56 > > diff -u -r1.56 pcb.h > > --- i386/include/pcb.h 29 Dec 2005 13:23:48 -0000 1.56 > > +++ i386/include/pcb.h 24 Apr 2008 06:46:59 -0000 > > @@ -81,7 +81,7 @@ > > struct trapframe; > > =20 > > void makectx(struct trapframe *, struct pcb *); > > -void savectx(struct pcb *); > > +int savectx(struct pcb *); > > #endif > > =20 > > #endif /* _I386_PCB_H_ */ > > Index: dev/acpica/acpi.c > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > RCS file: /home/ncvs/src/sys/dev/acpica/acpi.c,v > > retrieving revision 1.247 > > diff -u -r1.247 acpi.c > > --- dev/acpica/acpi.c 13 Mar 2008 20:39:03 -0000 1.247 > > +++ dev/acpica/acpi.c 30 Apr 2008 13:14:48 -0000 > > @@ -29,7 +29,7 @@ > > =20 > > #include > > __FBSDID("$FreeBSD: src/sys/dev/acpica/acpi.c,v 1.247 2008/03/13 20:39= :03 jhb Exp $"); > > - > > +#define SMP > > #include "opt_acpi.h" > > #include > > #include > > @@ -47,6 +47,7 @@ > > #include > > #include > > #include > > +#include > > =20 > > #include > > #include > > @@ -2339,6 +2340,8 @@ > > * drivers need this. > > */ > > mtx_lock(&Giant); > > + sched_bind(curthread, 0); > > + stop_cpus(PCPU_GET(other_cpus)); > > slp_state =3D ACPI_SS_NONE; > > switch (state) { > > case ACPI_STATE_S1: > > @@ -2430,13 +2433,16 @@ > > acpi_wake_prep_walk(state); > > sc->acpi_sstate =3D ACPI_STATE_S0; > > } > > + printf("PREP WALK\n"); > > if (slp_state >=3D ACPI_SS_SLP_PREP) > > AcpiLeaveSleepState(state); > > + printf("LEAVE_SLEEP_STATE\n"); > > if (slp_state >=3D ACPI_SS_DEV_SUSPEND) > > DEVICE_RESUME(root_bus); > > + printf("DEVICE_RESUME\n"); > > if (slp_state >=3D ACPI_SS_SLEPT) > > acpi_enable_fixed_events(sc); > > - > > + printf("ENABLE_FIXED_EVENT\n"); > > /* Allow another sleep request after a while. */ > > if (state !=3D ACPI_STATE_S5) > > timeout(acpi_sleep_enable, sc, hz * ACPI_MINIMUM_AWAKETIME); > > @@ -2445,6 +2451,7 @@ > > acpi_UserNotify("Resume", ACPI_ROOT_OBJECT, state); > > =20 > > mtx_unlock(&Giant); > > + sched_unbind(curthread); > > return_ACPI_STATUS (status); > > } > > =20 > > Index: dev/acpica/acpi_ec.c > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > RCS file: /home/ncvs/src/sys/dev/acpica/acpi_ec.c,v > > retrieving revision 1.80 > > diff -u -r1.80 acpi_ec.c > > --- dev/acpica/acpi_ec.c 8 Nov 2007 21:20:34 -0000 1.80 > > +++ dev/acpica/acpi_ec.c 7 May 2008 17:07:11 -0000 > > @@ -747,7 +747,7 @@ > > * If booting, check if we need to run the query handler. If so, = we > > * we call it directly here since our thread taskq is not active y= et. > > */ > > - if (cold || rebooting) { > > + if (cold || rebooting||sc->ec_suspending) { > > if ((EC_GET_CSR(sc) & EC_EVENT_SCI)) { > > CTR0(KTR_ACPI, "ec running gpe handler directly"); > > EcGpeQueryHandler(sc); > > _______________________________________________ > > freebsd-acpi@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-acpi > > To unsubscribe, send any mail to "freebsd-acpi-unsubscribe@freebsd.org" --=20 Vladimir B. Grebenschikov vova@fbsd.ru