From owner-freebsd-xen@FreeBSD.ORG Wed Jun 17 10:55:23 2009 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D9E81065688 for ; Wed, 17 Jun 2009 10:55:23 +0000 (UTC) (envelope-from prvs=alex.urbanowicz/0419299677@artegence.com) Received: from pciuch.3dart.com (pciuch.3dart.com [195.164.48.69]) by mx1.freebsd.org (Postfix) with ESMTP id 7CA068FC0A for ; Wed, 17 Jun 2009 10:55:22 +0000 (UTC) (envelope-from prvs=alex.urbanowicz/0419299677@artegence.com) Received: from worker3.non.3dart.com ([192.168.240.83]) by pciuch.3dart.com with esmtpsa (Cipher TLSv1:AES256-SHA:256) (Exim 4.67 #1) id 1MGsbo-0007kL-2B by authid with plain_login for ; Wed, 17 Jun 2009 12:43:12 +0200 From: Alex Urbanowicz To: freebsd-xen@freebsd.org In-Reply-To: <20090615120038.79B9F106580B@hub.freebsd.org> References: <20090615120038.79B9F106580B@hub.freebsd.org> Content-Type: multipart/mixed; boundary="=-XPogi9Zb7q4/PkN7zQi+" Date: Wed, 17 Jun 2009 10:43:07 +0000 Message-Id: <1245235387.15565.1799.camel@hosaka> Mime-Version: 1.0 X-Mailer: Evolution 2.24.3 X-Spam-Score: -11.4 (-----------) X-Spam-Report: No, hits=-11.4 required=7.0 tests=ALL_TRUSTED, PCIUCH_AUTH autolearn=disabled version=3.2.5 Subject: my experience with FreeBSD Xen guest X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2009 10:55:23 -0000 --=-XPogi9Zb7q4/PkN7zQi+ Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello I've spent last four days trying to get a stable 8.0-CURRENT on a CentOS 5.3. I am not very versed in BSD, so some of the problems may come from applying generic Unix approach to BSD environment. * disk operations are unstable. I attach dmesg from boot of a freshly fscked filesystem (16G LVM on host), the two stack backtraces are routinely happening during boot sequence. Also massive operations like buildworld usually lead to more syscall errors and routine filesystem corruption * sometimes the guest just crashes to kernel debugger, I have not isolated any particular reason why it happens * during the four days I only managed to successfully compile kernel once and world twice, installworld always fails, the last one with following error: [...] =3D=3D=3D> secure (install) =3D=3D=3D> secure/lib (install) =3D=3D=3D> secure/lib/libcrypto (install) ".depend", line 19879: Unassociated shell command "=EF=BF=BD=EF=BF=BD=C9=AB= =EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD'=EF=BF=BDF/= ^=EF=BF=BD>=EF=BF=BD =E9=97=86=10=EF=BF=BD=EF=BF=BD=06=EF=BF=BD ".depend", line 19880: Need an operator ".depend", line 19881: Need an operator ".depend", line 19883: Missing dependency operator ".depend", line 19884: Need an operator No closing parenthesis in archive specification ".depend", line 19885: Error in archive specification: "=1B% =EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BDZ=EF=BF=BD=EF=BF=BDF=EF=BF=BD=EF=BF=BD= =EF=BF=BD=14=EF=BF=BD=EF=BF=BD_o=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=7Fo=EF= =BF=BDx;=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BDw,=EF=BF=BD=1D=EF=BF=BDF=DE=8E= =EF=BF=BD/=177=EF=BF=BD=CD=9A=EF=BF=BD=EF=BF=BD" No closing parenthesis in archive specification ".depend", line 19886: Error in archive specification: "=EF=BF=BD=EF=BF=BD= =EF=BF=BD=EF=BF=BD=EF=BF=BD" ".depend", line 19888: Need an operator ".depend", line 19889: Need an operator ".depend", line 19893: Need an operator ".depend", line 19894: Missing dependency operator No closing parenthesis in archive specification ".depend", line 19899: Error in archive specification: "6pH=EF=BF=BDEX=EF=BF=BD=EF=BF=BD=1C=EF=BF=BD=EF=BF=BD]=EF=BF=BD=EF=BF=BD= =EF=BF=BD=EF=BF=BD9=EF=BF=BD=1B=E4=9D=BA8=EF=BF=BD" ".depend", line 19910: Need an operator make: fatal errors encountered -- cannot continue *** Error code 1 Stop in /usr/src/secure/lib. *** Error code 1 Stop in /usr/src/secure. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. Also, during the mentioned buildworld the following syscall error happened: lock order reversal: 1st 0xc393b81c snaplk (snaplk) @ /usr/home/alex/bsd/sys/kern/vfs_vnops.c:295 2nd 0xc441037c ufs (ufs) @ /usr/home/alex/bsd/sys/ufs/ffs/ffs_snapshot.c:1587 KDB: stack backtrace: X_db_sym_numargs(c0356957,e4d2f640,c0116b95,c01072bb,c035979e,...) at X_db_sym_numargs+0x146 kdb_backtrace(c01072bb,c035979e,c36ccf10,c36ccbd0,e4d2f69c,...) at kdb_backtrace+0x29 witness_display_spinlock(c035979e,c441037c,c034c8dc,c36ccbd0,c0371652,...) = at witness_display_spinlock+0x75 witness_checkorder(c441037c,9,c0371652,633,0,...) at witness_checkorder +0x839 __lockmgr_args(c441037c,80000,0,0,0,...) at __lockmgr_args+0x797 ffs_snapremove(c4410324,0,e4d2f7d8,e4d2f7d8,c035e67e,...) at ffs_snapremove+0x11f ffs_truncate(c4410324,0,0,c00,0,...) at ffs_truncate+0x5f7 ufs_inactive(e4d2fa58,c4410398,c4410324,c4410398,e4d2fa70,...) at ufs_inactive+0x218 VOP_INACTIVE_APV(c03b0200,e4d2fa58,c0360aae,925,c03c2160,...) at VOP_INACTIVE_APV+0xa5 vref(c03b0200,e4d2fa8c,c0360aae,8ab,127,...) at vref+0x136e vput(c4410324,e4d2fac8,c0361c78,127,0,...) at vput+0x1cd vn_close(c4410324,1,c3b02600,c3b1f240,c0c74880,...) at vn_close+0x19a vn_close(c39611f8,c3b1f240,3,0,c39611f8,...) at vn_close+0x294 _fdrop(c39611f8,c3b1f240,e4d2fb94,c01169dc,0,c0507a20,c3b1f2e4,c0507a20,c03= 9b680,c4bd432c,6f4,c034e229,e4d2fbbc,c00de690,c4bd432c,8,c034e229,6f4) at _= fdrop+0x43 closef(c39611f8,c3b1f240,6f4,6f1,c3b1f2e4,...) at closef+0x290 fdfree(c3b1f240,0,c034eafc,103,1,...) at fdfree+0x3ea exit1(c3b1f240,0,e4d2fd3c,c0329e23,c3b1f240,...) at exit1+0x533 sys_exit(c3b1f240,e4d2fd08,4,c035a0f3,c039227c,...) at sys_exit+0x1d syscall(e4d2fd48) at syscall+0x2a3 Xint0x80_syscall() at Xint0x80_syscall+0x22 --- syscall (1, FreeBSD ELF32, sys_exit), eip =3D 0x28129f63, esp =3D 0xbf7fe33c, ebp =3D 0xbf7fe348 --- I guess that the both of the errors are related to fs corruption that happened during the course of buildworld (before I started buildworld I launched the guest from different image and fscked the / clean). The filesystem problems are severe, and block us from using FreeBSD/Xen. Regards Alex --=-XPogi9Zb7q4/PkN7zQi+ Content-Disposition: attachment; filename="dmesg" Content-Type: text/plain; name="dmesg"; charset="UTF-8" Content-Transfer-Encoding: 7bit Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-CURRENT #0 r: Tue Jun 16 13:53:39 UTC 2009 alex@dev32:/usr/obj/usr/home/alex/bsd/sys/XEN WARNING: WITNESS option enabled, expect reduced performance. Xen reported: 2327.498 MHz processor. Timecounter "ixen" frequency 1000000000 Hz quality 0 CPU: Intel(R) Xeon(R) CPU E5410 @ 2.33GHz (2327.50-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x1067a Stepping = 10 Features=0xbfebfbff Features2=0x40ce3bd AMD Features=0x20000000 AMD Features2=0x1 TSC: P-state invariant Instruction TLB: 4 KB Pages, 4-way set associative, 128 entries 1st-level instruction cache: 32 KB, 8-way set associative, 64 byte line size 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size L2 cache: 6144 kbytes, 16-way associative, 64 bytes/line real memory = 1073741824 (1024 MB) Physical memory chunk(s): 0x0000000000895000 - 0x000000003edb2fff, 1045553152 bytes (255262 pages) avail memory = 1041563648 (993 MB) APIC: Using the MPTable enumerator. SMP: Added CPU 0 (BSP) ULE: setup cpu 0 [XEN] IPI cpu=0 irq=128 vector=RESCHEDULE_VECTOR (0) [XEN] IPI cpu=0 irq=129 vector=CALL_FUNCTION_VECTOR (1) Event-channel device installed. mem: Pentium Pro MTRR support enabled null: nfslock: pseudo-device random: io: kbd0 at kbdmux0 Grant table initialized xenbus0: on motherboard xc0: on motherboard [XEN] xen_rtc_probe: probing Hypervisor RTC clock rtc0: on motherboard [XEN] xen_rtc_attach: attaching Hypervisor RTC clock rtc0: registered as a time-of-day clock (resolution 1000000us) npx0: INT 16 interface Device configuration finished. procfs registered Timecounters tick every 10.000 msec [XEN] hypervisor wallclock nudged; nudging TOD. lo0: bpf attached xbd0: 16384MB at device/vbd/51712 on xenbus0 GEOM: new disk xbd0 xbd2: 256MB at device/vbd/51744 on xenbus0 xn0: at device/vif/0 on xenbus0 xn0: bpf attached xn0: Ethernet address: 00:bd:c4:13:00:01 WARNING: WITNESS option enabled, expect reduced performance. GEOM: new disk xbd2 Trying to mount root from ufs:xbd0 rtc0: [XEN] xen_rtc_gettime rtc0: [XEN] xen_rtc_gettime: wallclock 1241776380 sec; 699847734 nsec rtc0: [XEN] xen_rtc_gettime: uptime 3384929 sec; 293434434 nsec rtc0: [XEN] xen_rtc_gettime: TOD 1245161309 sec; 993282168 nsec start_init: trying /sbin/init lock order reversal: 1st 0xc3a8b8b8 ufs (ufs) @ /usr/home/alex/bsd/sys/kern/vfs_mount.c:1054 2nd 0xc3a8d6a0 devfs (devfs) @ /usr/home/alex/bsd/sys/kern/vfs_subr.c:2084 KDB: stack backtrace: X_db_sym_numargs(c0356957,c35b8824,c0116b95,c01072bb,c035979e,...) at X_db_sym_numargs+0x146 kdb_backtrace(c01072bb,c035979e,c36ccbd0,c36ccb00,c35b8880,...) at kdb_backtrace+0x29 witness_display_spinlock(c035979e,c3a8d6a0,c03489cb,c36ccb00,c0360aae,...) at witness_display_spinlock+0x75 witness_checkorder(c3a8d6a0,9,c0360aae,824,0,...) at witness_checkorder+0x839 __lockmgr_args(c3a8d6a0,80100,c3a8d6bc,0,0,...) at __lockmgr_args+0x797 vop_stdlock(c35b8988,c011693b,c0348c10,80100,c3a8d648,...) at vop_stdlock+0x62 VOP_LOCK1_APV(c038f2e0,c35b8988,c39c2524,c03c21a0,c3a8d648,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c3a8d648,80100,c0360aae,824,8,...) at _vn_lock+0x5e vget(c3a8d648,80100,c39c2480,15d,c0348b28,...) at vget+0xb9 devfs_allocv(c3952000,c394ca10,c35b8a20,9d,c0534698,...) at devfs_allocv+0x102 devfs_rules_apply(c394ca10,80000,c35b8c3c,42c,0,...) at devfs_rules_apply+0x14a vfs_donmount(c39c2480,0,c3952100,c3952100,bf7fde39,...) at vfs_donmount+0x14bb nmount(c39c2480,c35b8d08,c,c,c0394bb8,...) at nmount+0x72 syscall(c35b8d48) at syscall+0x2a3 Xint0x80_syscall() at Xint0x80_syscall+0x22 --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280e7c3b, esp = 0xbf7fde0c, ebp = 0xbf7fe368 --- lock order reversal: 1st 0xd772de30 bufwait (bufwait) @ /usr/home/alex/bsd/sys/kern/vfs_bio.c:2558 2nd 0xc3991e00 dirhash (dirhash) @ /usr/home/alex/bsd/sys/ufs/ufs/ufs_dirhash.c:285 KDB: stack backtrace: X_db_sym_numargs(c0356957,e4d2fa8c,c0116b95,c01072bb,c035979e,...) at X_db_sym_numargs+0x146 kdb_backtrace(c01072bb,c035979e,c36ca4d0,c36ccc38,e4d2fae8,...) at kdb_backtrace+0x29 witness_display_spinlock(c035979e,c3991e00,c0373b42,c36ccc38,c03737d1,...) at witness_display_spinlock+0x75 witness_checkorder(c3991e00,9,c03737d1,11d,0,...) at witness_checkorder+0x839 _sx_xlock(c3991e00,0,c03737d1,11d,d86a3018,...) at _sx_xlock+0x85 ufsdirhash_enduseful(0,d,c38c4800,d772ddd0,d86a3018,...) at ufsdirhash_enduseful+0x2f5 ufsdirhash_remove(c3b2a0e8,d86a3018,18,e4d2fb78,e4d2fb74,...) at ufsdirhash_remove+0x14 ufs_dirremove(c3b25b84,c3b291d0,500800c,1,9f,...) at ufs_dirremove+0xe5 ufs_itimes(e4d2fc44,e4d2fc54,c3b20860,c3b20860,0,...) at ufs_itimes+0x826 VOP_RMDIR_APV(c03b0200,e4d2fc44,2,0,28216238,...) at VOP_RMDIR_APV+0xa5 kern_rmdirat(c3b16000,ffffff9c,28216238,0,e4d2fc90,...) at kern_rmdirat+0x16b kern_rmdir(c3b16000,28216238,0,e4d2fd3c,c0329e23,...) at kern_rmdir+0x27 rmdir(c3b16000,e4d2fd08,4,c036e7d5,c039315c,...) at rmdir+0x22 syscall(e4d2fd48) at syscall+0x2a3 Xint0x80_syscall() at Xint0x80_syscall+0x22 --- syscall (137, FreeBSD ELF32, rmdir), eip = 0x280d8bfb, esp = 0xbf7febfc, ebp = 0xbf7fec28 --- --=-XPogi9Zb7q4/PkN7zQi+--