From owner-freebsd-xen@freebsd.org Tue Oct 20 15:14:45 2015 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8B4DA1A9FD for ; Tue, 20 Oct 2015 15:14:45 +0000 (UTC) (envelope-from lars@netapp.com) Received: from mx142.netapp.com (mx142.netapp.com [216.240.21.19]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx142.netapp.com", Issuer "Symantec Class 3 Secure Server CA - G4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8156E27D; Tue, 20 Oct 2015 15:14:45 +0000 (UTC) (envelope-from lars@netapp.com) X-IronPort-AV: E=Sophos;i="5.17,707,1437462000"; d="asc'?scan'208";a="71901659" Received: from hioexcmbx04-prd.hq.netapp.com ([10.122.105.37]) by mx142-out.netapp.com with ESMTP; 20 Oct 2015 08:13:36 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com (10.122.105.40) by hioexcmbx04-prd.hq.netapp.com (10.122.105.37) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Tue, 20 Oct 2015 08:13:36 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com ([::1]) by hioexcmbx07-prd.hq.netapp.com ([fe80::e1d9:911e:3048:d510%21]) with mapi id 15.00.1104.000; Tue, 20 Oct 2015 08:13:36 -0700 From: "Eggert, Lars" To: =?Windows-1252?Q?Roger_Pau_Monn=E9?= CC: Roman Bogorodskiy , "freebsd-xen@freebsd.org" Subject: Re: Xen dom0 crash Thread-Topic: Xen dom0 crash Thread-Index: AQHQkWsjWWZFxg5CcECFAUy3VuNwPJ2CMi0AgAZ0NYCAAARrAIAFs0gAgAABZICA54bPAA== Date: Tue, 20 Oct 2015 15:13:35 +0000 Message-ID: References: <7CB114B9-B666-4764-86B6-B94F07973389@netapp.com> <5559EB2E.6080609@citrix.com> <20150522161117.GA59488@kloomba> <555F58DC.2040807@citrix.com> <46625CAE-750E-4ECE-8B39-DF8EDDFA912F@netapp.com> <55642229.9020106@citrix.com> In-Reply-To: <55642229.9020106@citrix.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-mailer: Apple Mail (2.3094) x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.122.56.79] Content-Type: multipart/signed; boundary="Apple-Mail=_0160A8EF-9FF6-45F7-AF1F-12E86064B89B"; protocol="application/pgp-signature"; micalg=pgp-sha256 MIME-Version: 1.0 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2015 15:14:45 -0000 --Apple-Mail=_0160A8EF-9FF6-45F7-AF1F-12E86064B89B Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Hi, with a recent -CURRENT and xen-4.6.0, I now see a different crash at = boot on the same box. Any ideas? Thanks, Lars = B2 ______ ____ _____ _____ | ____| | _ \ / ____| __ \ | |___ _ __ ___ ___ | |_) | (___ | | | | | ___| '__/ _ \/ _ \| _ < \___ \| | | | | | | | | __/ __/| |_) |____) | |__| | | | | | | | || | | | Xen 4.6.0el/cc_vegas.ko size 0x3348 at 0x1403000 = ` (XEN) Xen version 4.6.0 (root@netapp.com) (gcc47 (FreeBSD Ports = Collection) 4.7.4) debug=3Dn Wed Oct 14 18:50:20 CEST = 2015g...el/cc_hd.ko size 0x2ea8 at 0x13ff000o .--` /y:` = +. (XEN) Latest ChangeSet: 'ertt' | yo`:. :o = `+- (XEN) Bootloader: FreeBSD Loader | y/ -/` = -o/ (XEN) Command line: guest_loglvl=3Dall loglvl=3Dal dom0_max_vcpus=3D32 = dom0_mem=3D4096M dom0pvh=3D1 com1=3D115200,8n1 console=3Dcom13. [Esc]ape = to loader prompt | / `-- / (XEN) Video information: | `: = :` (XEN) VGA is text mode 80x25, font 8x16 | `: = :` (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds = / (XEN) EDID info not retrieved because no DDC retrieval method detected = -. (XEN) Disc information:O]ptions... | -- = -. (XEN) Found 1 MBR signatures | `:` `:` (XEN) Found 1 EDD information structures | .-- `--. (XEN) Xen-e820 RAM map: | .---.....----. (XEN) 0000000000000000 - 000000000009ac00 (usable) (XEN) 000000000009ac00 - 00000000000a0000 (reserved) (XEN) 00000000000e0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 000000007de19000 (usable) (XEN) 000000007de19000 - 000000007dea8000 (reserved) (XEN) 000000007dea8000 - 000000007dfb1000 (ACPI data) (XEN) 000000007dfb1000 - 000000007e1cb000 (ACPI NVS) (XEN) 000000007e1cb000 - 000000007f355000 (reserved) (XEN) 000000007f355000 - 000000007f800000 (ACPI NVS) (XEN) 0000000080000000 - 0000000090000000 (reserved) (XEN) 00000000fed1c000 - 00000000fed40000 (reserved) (XEN) 00000000ff000000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 0000002080000000 (usable) (XEN) ACPI: RSDP 000F04A0, 0024 (r2 SUPERM) (XEN) ACPI: XSDT 7DEDD088, 008C (r1 SUPERM SMCI--MB 1 AMI = 10013) (XEN) ACPI: FACP 7DEE80F8, 00F4 (r4 SUPERM SMCI--MB 1 AMI = 10013) (XEN) ACPI: DSDT 7DEDD1A0, AF53 (r2 SUPERM SMCI--MB 0 INTL = 20091112) (XEN) ACPI: FACS 7E1C2080, 0040 (XEN) ACPI: APIC 7DEE81F0, 0224 (r3 1 AMI = 10013) (XEN) ACPI: FPDT 7DEE8418, 0044 (r1 1 AMI = 10013) (XEN) ACPI: HPET 7DEE8460, 0038 (r1 SUPERM SMCI--MB 1 AMI. = 5) (XEN) ACPI: PRAD 7DEE8498, 00BE (r2 PRADID PRADTID 1 MSFT = 4000000) (XEN) ACPI: SPMI 7DEE8558, 0040 (r5 A M I OEMSPMI 0 AMI. = 0) (XEN) ACPI: SSDT 7DEE8598, C7AE8 (r2 INTEL CpuPm 4000 INTL = 20091112) (XEN) ACPI: EINJ 7DFB0080, 0130 (r1 AMI AMI EINJ 0 = 0) (XEN) ACPI: ERST 7DFB01B0, 0230 (r1 AMIER AMI ERST 0 = 0) (XEN) ACPI: HEST 7DFB03E0, 00A8 (r1 AMI AMI HEST 0 = 0) (XEN) ACPI: BERT 7DFB0488, 0030 (r1 AMI AMI BERT 0 = 0) (XEN) ACPI: DMAR 7DFB04B8, 0168 (r1 A M I OEMDMAR 1 INTL = 1) (XEN) ACPI: MCFG 7DFB0620, 003C (r1 SUPERM SMCI--MB 1 MSFT = 97) (XEN) System RAM: 131037MB (134182604kB) (XEN) Domain heap initialised (XEN) ACPI: 32/64X FACS address mismatch in FADT - = 7e1c2080/0000000000000000, using 32 (XEN) Processor #0 6:13 APIC version 21 (XEN) Processor #2 6:13 APIC version 21 (XEN) Processor #4 6:13 APIC version 21 (XEN) Processor #6 6:13 APIC version 21 (XEN) Processor #8 6:13 APIC version 21 (XEN) Processor #10 6:13 APIC version 21 (XEN) Processor #12 6:13 APIC version 21 (XEN) Processor #14 6:13 APIC version 21 (XEN) Processor #32 6:13 APIC version 21 (XEN) Processor #34 6:13 APIC version 21 (XEN) Processor #36 6:13 APIC version 21 (XEN) Processor #38 6:13 APIC version 21 (XEN) Processor #40 6:13 APIC version 21 (XEN) Processor #42 6:13 APIC version 21 (XEN) Processor #44 6:13 APIC version 21 (XEN) Processor #46 6:13 APIC version 21 (XEN) Processor #1 6:13 APIC version 21 (XEN) Processor #3 6:13 APIC version 21 (XEN) Processor #5 6:13 APIC version 21 (XEN) Processor #7 6:13 APIC version 21 (XEN) Processor #9 6:13 APIC version 21 (XEN) Processor #11 6:13 APIC version 21 (XEN) Processor #13 6:13 APIC version 21 (XEN) Processor #15 6:13 APIC version 21 (XEN) Processor #33 6:13 APIC version 21 (XEN) Processor #35 6:13 APIC version 21 (XEN) Processor #37 6:13 APIC version 21 (XEN) Processor #39 6:13 APIC version 21 (XEN) Processor #41 6:13 APIC version 21 (XEN) Processor #43 6:13 APIC version 21 (XEN) Processor #45 6:13 APIC version 21 (XEN) Processor #47 6:13 APIC version 21 (XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23 (XEN) IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47 (XEN) IOAPIC[2]: apic_id 3, version 32, address 0xfec40000, GSI 48-71 (XEN) Enabling APIC mode: Phys. Using 3 I/O APICs (XEN) Switched to APIC driver x2apic_cluster. (XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7 (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Detected 2000.043 MHz processor. (XEN) Initing memory sharing. (XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB. (XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB. (XEN) Intel VT-d Snoop Control enabled. (XEN) Intel VT-d Dom0 DMA Passthrough not enabled. (XEN) Intel VT-d Queued Invalidation enabled. (XEN) Intel VT-d Interrupt Remapping enabled. (XEN) Intel VT-d Shared EPT tables enabled. (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Relaxed (XEN) Interrupt remapping enabled (XEN) Enabled directed EOI with ioapic_ack_old on! (XEN) ENABLING IO-APIC IRQs (XEN) -> Using old ACK method (XEN) Platform timer is 14.318MHz HPET (XEN) Allocated console ring of 64 KiB. (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Extended Page Tables (EPT) (XEN) - Virtual-Processor Identifiers (VPID) (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) - Unrestricted Guest (XEN) HVM: ASIDs enabled. (XEN) HVM: VMX enabled (XEN) HVM: Hardware Assisted Paging (HAP) detected (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB (XEN) Brought up 32 CPUs (XEN) Dom0 has maximum 1224 PIRQs (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x200000 -> 0x15ed678 (XEN) Dom0 symbol map 0x15ed678 -> 0x180ba40 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000002020000000->0000002024000000 (1032178 pages = to be allocated) (XEN) Init. ramdisk: 000000207fff2000->0000002080000000 (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80200000->ffffffff8180ba40 (XEN) Init. ramdisk: ffffffff8180c000->ffffffff8181a000 (XEN) Phys-Mach map: ffffffff8181a000->ffffffff8201a000 (XEN) Start info: ffffffff8201a000->ffffffff8201b4b4 (XEN) Page tables: ffffffff8201c000->ffffffff82031000 (XEN) Boot stack: ffffffff82031000->ffffffff82032000 (XEN) TOTAL: ffffffff80000000->ffffffff82400000 (XEN) ENTRY ADDRESS: ffffffff80963000 (XEN) Dom0 has maximum 32 VCPUs (XEN) Scrubbing Free RAM on 1 nodes using 16 CPUs (XEN) = .................................................................done. (XEN) Initial low memory virq threshold set at 0x4000 pages. (XEN) Std. Loglevel: Errors and warnings (XEN) Guest Loglevel: All (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch = input to Xen) (XEN) Freed 296kB init memory. FreeBSD PVH running on xen-3.0-x86_64p GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2015 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights = reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #15 384e927(muclab)-dirty: Tue Oct 20 16:46:27 CEST = 2015 = elars@laurel.muccbc.hq.netapp.com:/usr/home/elars/obj/usr/home/elars/src/s= ys/MUCLAB amd64 FreeBSD clang version 3.7.0 (tags/RELEASE_370/final 246257) 20150906 sysctl: OID number(10) is already in use for 'intr_direct_queue_maxlen' sysctl: OID number(11) is already in use for 'intr_direct_queue_drops' VT(vga): resolution 640x480 (XEN) Direct Vector 0x93 CPU: Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz (2000.00-MHz K8-class CPU) Origin=3D"GenuineIntel" Id=3D0x206d7 Family=3D0x6 Model=3D0x2d = Stepping=3D7 = Features=3D0x1fc3ebff = Features2=3D0x9fb82283 AMD Features=3D0x20100800 AMD Features2=3D0x1 XSAVE Features=3D0x1 TSC: P-state invariant, performance statistics Hypervisor: Origin =3D "XenVMMXenVMM" real memory =3D 6478413824 (6178 MB) avail memory =3D 4065890304 (3877 MB) FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs FreeBSD/SMP: 2 package(s) x 16 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 2 cpu2 (AP): APIC ID: 4 cpu3 (AP): APIC ID: 6 cpu4 (AP): APIC ID: 8 cpu5 (AP): APIC ID: 10 cpu6 (AP): APIC ID: 12 cpu7 (AP): APIC ID: 14 cpu8 (AP): APIC ID: 16 cpu9 (AP): APIC ID: 18 cpu10 (AP): APIC ID: 20 cpu11 (AP): APIC ID: 22 cpu12 (AP): APIC ID: 24 cpu13 (AP): APIC ID: 26 cpu14 (AP): APIC ID: 28 cpu15 (AP): APIC ID: 30 cpu16 (AP): APIC ID: 32 cpu17 (AP): APIC ID: 34 cpu18 (AP): APIC ID: 36 cpu19 (AP): APIC ID: 38 cpu20 (AP): APIC ID: 40 cpu21 (AP): APIC ID: 42 cpu22 (AP): APIC ID: 44 cpu23 (AP): APIC ID: 46 cpu24 (AP): APIC ID: 48 cpu25 (AP): APIC ID: 50 cpu26 (AP): APIC ID: 52 cpu27 (AP): APIC ID: 54 cpu28 (AP): APIC ID: 56 cpu29 (AP): APIC ID: 58 cpu30 (AP): APIC ID: 60 cpu31 (AP): APIC ID: 62 random: entropy device external interface module_register_init: MOD_LOAD (vesa, 0xffffffff80994b90, 0) error 19 netmap: loaded module vtvga0: on motherboard smbios0: at iomem 0xf04d0-0xf04ee on = motherboard smbios0: Version: 2.7, BCD Revision: 2.7 aesni0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 attimer0: Can't map interrupt. atrtc0: port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 (XEN) d0v0 EPT violation 0x1aa (-w-/r-x) gpa 0x000000800f8060 mfn = 0x800f8 type 5 (XEN) d0v0 Walking EPT tables for GFN 800f8: (XEN) d0v0 epte 80000020269b7007 (XEN) d0v0 epte 800000207fd8f007 (XEN) d0v0 epte 800000207fd8e007 (XEN) d0v0 epte 80500000800f8805 (XEN) d0v0 --- GLA 0xfffff800800f8060 (XEN) domain_crash called from vmx.c:2691 (XEN) Domain 0 (vcpu#0) crashed on cpu#7: (XEN) ----[ Xen-4.6.0 x86_64 debug=3Dn Not tainted ]---- (XEN) CPU: 7 (XEN) RIP: 0020:[] (XEN) RFLAGS: 0000000000010246 CONTEXT: hvm guest (d0v0) (XEN) rax: 000000000000008b rbx: 00000000000f8000 rcx: = 00000000000f8060 (XEN) rdx: fffff80080000000 rsi: 000000000000001f rdi: = 0000000000000000 (XEN) rbp: ffffffff82038390 rsp: ffffffff82038350 r8: = 000000000000008b (XEN) r9: 0000000000000001 r10: ffffffff802ffc80 r11: = fffff80005d1e830 (XEN) r12: 0000000000000001 r13: 00000000000f8000 r14: = 0000000000000000 (XEN) r15: 0000000000000060 cr0: 000000008005003b cr4: = 00000000000406e0 (XEN) cr3: 0000000002043000 cr2: 0000000000000000 (XEN) ds: 003b es: 003b fs: 0013 gs: 0028 ss: 0028 cs: 0020 (XEN) Guest stack trace from rsp=3Dffffffff82038350: (XEN) Fault while accessing guest memory. (XEN) Hardware Dom0 crashed: rebooting machine in 5 seconds. (XEN) Resetting with ACPI MEMORY or I/O RESET_REG. --Apple-Mail=_0160A8EF-9FF6-45F7-AF1F-12E86064B89B Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQCVAwUBViZaH9ZcnpRveo1xAQg3tgQAhjPvS/xBb/UERbVaw5oFh2dHfeGepou+ g61H4PgMq4sbeG5taykvICTnqVMdri9Lip3HUmgE90WU5jLfa4whfMevGM2sScra vb1kJguwEPjjVROZ9k6qu+glDOn5DE6ReOQhWZK6Eyr+P4lpiT05g7ZerTRnK0YD HQK9mOvZJ1k= =mYvQ -----END PGP SIGNATURE----- --Apple-Mail=_0160A8EF-9FF6-45F7-AF1F-12E86064B89B--