From owner-freebsd-current@FreeBSD.ORG Tue Feb 17 20:30:19 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CBC66C74; Tue, 17 Feb 2015 20:30:19 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AAA2FE8F; Tue, 17 Feb 2015 20:30:19 +0000 (UTC) Received: from [192.168.200.212] (unknown [50.136.155.142]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 7D432192A3B; Tue, 17 Feb 2015 20:30:18 +0000 (UTC) Message-ID: <54E3A4D9.4080604@ignoranthack.me> Date: Tue, 17 Feb 2015 12:30:17 -0800 From: Sean Bruno Reply-To: sbruno@freebsd.org User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: kib@freebsd.org Subject: Re: Xen HVM Panic, HEAD References: <54E2BF1E.4090002@ignoranthack.me> <20150217085657.GR34251@kib.kiev.ua> <54E39DC4.4010802@ignoranthack.me> <20150217202653.GW34251@kib.kiev.ua> In-Reply-To: <20150217202653.GW34251@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Cc: FreeBSD CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Feb 2015 20:30:19 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 02/17/15 12:26, Konstantin Belousov wrote: > On Tue, Feb 17, 2015 at 12:00:04PM -0800, Sean Bruno wrote: >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 >> >> On 02/17/15 00:56, Konstantin Belousov wrote: >>> On Mon, Feb 16, 2015 at 08:10:06PM -0800, Sean Bruno wrote: >>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 >>>> >>>> https://people.freebsd.org/~sbruno/Xen_APIC_panic.png >>>> >>>> I suspect that there may be one or two more lines above this >>>> that are relevant to this panic, but XENHVM kernel's now >>>> panic booting on Xen server. The working kernel output looks >>>> like this: >>>> >>>> FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final >>>> 208032) 20140512 XEN: Hypervisor version 4.2 detected. CPU: >>>> Intel(R) Xeon(R) CPU E5620 @ 2.40GHz (2400.05-MHz >>>> K8-class CPU) Origin="GenuineIntel" Id=0x206c2 Family=0x6 >>>> Model=0x2c Stepping=2 >>>> Features=0x1783fbff >>>> >>>> >> >>>> Features2=0x81ba2201 >>>> AMD Features=0x28100800 AMD >>>> Features2=0x1 Hypervisor: Origin = "XenVMMXenVMM" real >>>> memory = 1434451968 (1368 MB) avail memory = 1353293824 >>>> (1290 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: >>>> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs >>>> FreeBSD/SMP: 1 package(s) x 2 core(s) cpu0 (BSP): APIC ID: 0 >>>> cpu1 (AP): APIC ID: 2 ioapic0: Changing APIC ID to 1 MADT: >>>> Forcing active-low polarity and level trigger for SCI >>> I am not sure why your machine uses native lapic instead of >>> xen lapic, and should it be other way, or not. >>> >>> Regardless, show the line number for the ipi_startup+0x56. Did >>> you performed clean kernel build ? >>> >>> >> >> I have rebuilt a kernel/world based on head at svn r276627. I >> have delete /usr/obj completely and started from scratch. >> >> Updated kernelpanic image at >> https://people.freebsd.org/~sbruno/Xen_APIC_panic.png >> >> /usr/src/sys/x86/include # kgdb /boot/kernel/kernel GNU gdb 6.1.1 >> [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is >> free software, covered by the GNU General Public License, and you >> are welcome to change it and/or distribute copies of it under >> certain conditions. Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" >> for details. This GDB was configured as >> "amd64-marcel-freebsd"... (kgdb) list *(ipi_startup+0x56) >> 0xffffffff80e088c6 is in ipi_startup (apicvar.h:383). 378 379 >> static inline int 380 lapic_ipi_wait(int delay) 381 { 382 383 >> return (apic_ops.ipi_wait(delay)); 384 } 385 386 static inline >> int 387 lapic_set_lvt_mask(u_int apic_id, u_int lvt, u_char >> masked) >> > > Please disassemble your ipi_startup, also please do 'p *apic_ops'. > > (kgdb) disassemble ipi_startup Dump of assembler code for function ipi_startup: 0xffffffff80df3900 : push %rbp 0xffffffff80df3901 : mov %rsp,%rbp 0xffffffff80df3904 : push %r14 0xffffffff80df3906 : push %rbx 0xffffffff80df3907 : mov %esi,%ebx 0xffffffff80df3909 : mov %edi,%r14d 0xffffffff80df390c : mov $0xc500,%edi 0xffffffff80df3911 : mov %r14d,%esi 0xffffffff80df3914 : callq *0xffffffff815ac428 0xffffffff80df391b : mov $0x14,%edi 0xffffffff80df3920 : callq *0xffffffff815ac438 0xffffffff80df3927 : mov $0x8500,%edi 0xffffffff80df392c : mov %r14d,%esi 0xffffffff80df392f : callq *0xffffffff815ac428 0xffffffff80df3936 : mov $0x2710,%edi 0xffffffff80df393b : callq 0xffffffff80f39c10 0xffffffff80df3940 : or $0x4600,%ebx 0xffffffff80df3946 : movslq %ebx,%rbx 0xffffffff80df3949 : mov %rbx,%rdi 0xffffffff80df394c : mov %r14d,%esi 0xffffffff80df394f : callq *0xffffffff815ac428 0xffffffff80df3956 : mov $0x14,%edi 0xffffffff80df395b : callq *0xffffffff815ac438 0xffffffff80df3962 : test %eax,%eax 0xffffffff80df3964 : je 0xffffffff80df399b 0xffffffff80df3966 : mov $0xc8,%edi 0xffffffff80df396b : callq 0xffffffff80f39c10 0xffffffff80df3970 : mov %rbx,%rdi 0xffffffff80df3973 : mov %r14d,%esi 0xffffffff80df3976 : callq *0xffffffff815ac428 0xffffffff80df397d : mov $0x14,%edi 0xffffffff80df3982 : callq *0xffffffff815ac438 0xffffffff80df3989 : test %eax,%eax 0xffffffff80df398b : je 0xffffffff80df39a4 0xffffffff80df398d : mov $0xc8,%edi 0xffffffff80df3992 : pop %rbx 0xffffffff80df3993 : pop %r14 0xffffffff80df3995 : pop %rbp 0xffffffff80df3996 : jmpq 0xffffffff80f39c10 0xffffffff80df399b : mov $0xffffffff810cb5c4,%rdi 0xffffffff80df39a2 : jmp 0xffffffff80df39ab 0xffffffff80df39a4 : mov $0xffffffff810cb5f3,%rdi 0xffffffff80df39ab : xor %eax,%eax 0xffffffff80df39ad : mov %r14d,%esi 0xffffffff80df39b0 : callq 0xffffffff809b40d0 End of assembler dump. (kgdb) p apic_ops $2 = {create = 0xffffffff80f32e10 , init = 0xffffffff80f32f70 , xapic_mode = 0xffffffff80f331b0 , setup = 0xffffffff80f331e0 , dump = 0xffffffff80f336c0 , disable = 0xffffffff80f339a0 , eoi = 0xffffffff80f33a00 , id = 0xffffffff80f33a40 , intr_pending = 0xffffffff80f33a80 , set_logical_id = 0xffffffff80f33ad0 , cpuid = 0xffffffff80f33b10 , alloc_vector = 0xffffffff80f33b20 , alloc_vectors = 0xffffffff80f33c90 , enable_vector = 0xffffffff80f33ec0 , disable_vector = 0xffffffff80f33ef0 , free_vector = 0xffffffff80f33f00 , enable_pmc = 0xffffffff80f34110 , disable_pmc = 0xffffffff80f34200 , reenable_pmc = 0xffffffff80f34260 , enable_cmc = 0xffffffff80f342c0 , ipi_raw = 0xffffffff80f34310 , ipi_vectored = 0xffffffff80f343d0 , ipi_wait = 0xffffffff80f34470 , set_lvt_mask = 0xffffffff80f34550 , set_lvt_mode = 0xffffffff80f34650 , set_lvt_polarity = 0xffffffff80f347e0 , set_lvt_triggermode = 0xffffffff80f348f0 } -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAEBCgBmBQJU46TUXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRCQUFENDYzMkU3MTIxREU4RDIwOTk3REQx MjAxRUZDQTFFNzI3RTY0AAoJEBIB78oecn5kX4QH/1NDTE1MSD0ni/UbPqBKYTFd Q6kgOVgX+fanXbI/dYGbZN0wFytxRl83Rm0bfZXrL8aUEJmFXDetsJDky5hKPl+2 F9nZXo3KOKgoAQ0K96rE+MUiKe1lQuSfxZggvMZQd6lTHSJDPriLytEPAf2DIOYk +31AtLmXlyx10yaRfhMBdMI7KDcIf22p8rU3eOv0xCK11QthXIohAbdfjW4RkXl1 VzF3VZ6TdJksvpDihREOfAkTCCMjeERCLT8lSQPBfogGIU0E7LE1SqfCQ9jagTBz nDCDyVDkpJT91Dn3FiCKmbOi7Rw78DAVMYBS+riWVXC64wKrXpqgg85+1npU4so= =odRC -----END PGP SIGNATURE-----