From owner-freebsd-current@FreeBSD.ORG Sat Aug 23 01:08:54 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A58FB1065672 for ; Sat, 23 Aug 2008 01:08:54 +0000 (UTC) (envelope-from pitney.brad@googlemail.com) Received: from wf-out-1314.google.com (wf-out-1314.google.com [209.85.200.175]) by mx1.freebsd.org (Postfix) with ESMTP id 711AC8FC18 for ; Sat, 23 Aug 2008 01:08:54 +0000 (UTC) (envelope-from pitney.brad@googlemail.com) Received: by wf-out-1314.google.com with SMTP id 24so604413wfg.7 for ; Fri, 22 Aug 2008 18:08:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=AFxejyeDNWowdGFw8s+/bsXq+y11YE2i0MVXBz4ZpwE=; b=dfMQ35gaJ6uYwTm02j37Il2ZGRrkUmT5daSr1ZsYuAsUV9fKhbga7KowTeBV1Eutsh a119UiTBlNNJdVgWxlRHyV2vKGuPOW1FFa0tqZECo1B92Iwm++pAy+ef78uFIPRSMHRA /ubREznmr+fhVYqjySl6ar9nYgsD13Ldjur4s= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=EqkuXpjlveurk7CoY/ZhyoaFZ5ON/1sYfUIbDE/AcdAU2tQzQWdJHJ6fRsRt1QBKuP DNBoj8Z+o7E/Am8EUMxERC3FubekyRPhRfme1iFM+/+tk2+Ke00/M90N4hi5U+4bHDuX RZiMGqxjllkg+XsmogT3aVBh/iTvEy5of5f50= Received: by 10.143.2.19 with SMTP id e19mr633592wfi.90.1219453734004; Fri, 22 Aug 2008 18:08:54 -0700 (PDT) Received: by 10.142.105.2 with HTTP; Fri, 22 Aug 2008 18:08:53 -0700 (PDT) Message-ID: <3dd203290808221808m2cff2e50tef3b3407628dea17@mail.gmail.com> Date: Sat, 23 Aug 2008 02:08:53 +0100 From: "Brad Pitney" To: "Kip Macy" In-Reply-To: <3c1674c90808221625gda3be13gf63a71666876cfef@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <3dd203290808221612r338afd77h7397595eb48ce6a7@mail.gmail.com> <3c1674c90808221625gda3be13gf63a71666876cfef@mail.gmail.com> X-Mailman-Approved-At: Sat, 23 Aug 2008 02:53:47 +0000 Cc: freebsd-current@freebsd.org Subject: Re: [XEN] xs_single device/vbd xs_talkv read_reply panic: bad stray interrupt X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Aug 2008 01:08:54 -0000 On Sat, Aug 23, 2008 at 12:25 AM, Kip Macy wrote: > I just committed a band-aid for this. Please try. > > Thanks, > Kip > thanks for the quick reply & fix! dmesg: WARNING: loader(8) metadata is missing! GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2008 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-CURRENT #0: Sat Aug 23 00:29:27 UTC 2008 root@test:/usr/obj/usr/src/sys/XEN WARNING: WITNESS option enabled, expect reduced performance. Xen reported: 1799.489 MHz processor. Timecounter "ixen" frequency 1000000000 Hz quality 0 CPU: AMD Sempron(tm) Processor 3000+ (1799.49-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x20fc2 Stepping = 2 Features=0x78bfbff Features2=0x1 AMD Features=0xe2500800 AMD Features2=0x1 real memory = 268435456 (256 MB) avail memory = 255016960 (243 MB) kbd0 at kbdmux0 xc0: on motherboard Timecounters tick every 10.000 msec [XEN] Initialising virtual ethernet driver. xs_single device xs_talkv read_reply event=1 itype=1 htype=1 len=16 xs_single device/vbd xs_talkv read_reply event=1 itype=1 htype=1 len=6 gather xs_read xs_single device/vbd/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 1 gather xs_read xs_single device/vbd/1/backend-id xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 0 xs_read xs_single device/vbd/1/backend xs_talkv read_reply event=2 itype=2 htype=2 len=NULL /local/domain/0/backend/vbd/16/1 xs_single /local/domain/0/backend/vbd/16/1 xs_talkv read_reply event=1 itype=1 htype=1 len=127 xs_read xs_single device/vbd/1/virtual-device xs_talkv read_reply event=2 itype=2 htype=2 len=NULL xs_single xs_talkv read_reply event=6 itype=6 htype=6 len=NULL xenbus_write dir=device/vbd/1 val=8 xs_talkv read_reply event=11 itype=11 htype=11 len=NULL xenbus_write dir=device/vbd/1 val=4 xs_talkv read_reply event=11 itype=11 htype=11 len=NULL xenbus_transaction_end xs_single T xs_talkv read_reply event=7 itype=7 htype=7 len=NULL xs_read xs_single device/vbd/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL xenbus_write dir=device/vbd/1 val=3 xs_talkv read_reply event=11 itype=11 htype=11 len=NULL xs_talkv read_reply event=4 itype=4 htype=4 len=NULL gather xs_read xs_single device/vbd/2/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/sectors xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 2097152 xs_read xs_single /local/domain/0/backend/vbd/16/1/info xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 0 xs_read xs_single /local/domain/0/backend/vbd/16/1/sector-size xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 512 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/feature-barrier xs_talkv read_reply event=16 itype=2 htype=16 len=NULL xs_read xs_single device/vbd/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL xenbus_write dir=device/vbd/1 val=4 xs_talkv read_reply event=11 itype=11 htype=11 len=NULL 1 gather xs_read xs_single device/vbd/2/backend-id xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 0 xs_read xs_single device/vbd/2/backend xs_talkv read_reply event=2 itype=2 htype=2 len=NULL /local/domain/0/backend/vbd/16/2 xs_single /local/domain/0/backend/vbd/16/2 xs_talkv read_reply event=1 itype=1 htype=1 len=127 xs_read xs_single device/vbd/2/virtual-device xs_talkv read_reply event=2 itype=2 htype=2 len=NULL xs_single xs_talkv read_reply event=6 itype=6 htype=6 len=NULL xenbus_write dir=device/vbd/2 val=9 xs_talkv read_reply event=11 itype=11 htype=11 len=NULL xenbus_write dir=device/vbd/2 val=5 xs_talkv read_reply event=11 itype=11 htype=11 len=NULL xenbus_transaction_end xs_single T xs_talkv read_reply event=7 itype=7 htype=7 len=NULL xs_read xs_single device/vbd/2/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL xenbus_write dir=device/vbd/2 val=3 xs_talkv read_reply event=11 itype=11 htype=11 len=NULL xs_talkv read_reply event=4 itype=4 htype=4 len=NULL gather xs_read xs_single device/vbd/3/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL gather xs_read xs_single /local/domain/0/backend/vbd/16/2/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/2/sectors xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 1048576 xs_read xs_single /local/domain/0/backend/vbd/16/2/info xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 0 xs_read xs_single /local/domain/0/backend/vbd/16/2/sector-size xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 512 gather xs_read xs_single /local/domain/0/backend/vbd/16/2/feature-barrier xs_talkv read_reply event=16 itype=2 htype=16 len=NULL xs_read xs_single device/vbd/2/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL xenbus_write dir=device/vbd/2 val=4 xs_talkv read_reply event=11 itype=11 htype=11 len=NULL 1 gather xs_read xs_single device/vbd/3/backend-id xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 0 xs_read xs_single device/vbd/3/backend xs_talkv read_reply event=2 itype=2 htype=2 len=NULL /local/domain/0/backend/vbd/16/3 xs_single /local/domain/0/backend/vbd/16/3 xs_talkv read_reply event=1 itype=1 htype=1 len=127 xs_read xs_single device/vbd/3/virtual-device xs_talkv read_reply event=2 itype=2 htype=2 len=NULL xs_single xs_talkv read_reply event=6 itype=6 htype=6 len=NULL xenbus_write dir=device/vbd/3 val=10 xs_talkv read_reply event=11 itype=11 htype=11 len=NULL xenbus_write dir=device/vbd/3 val=6 xs_talkv read_reply event=11 itype=11 htype=11 len=NULL xenbus_transaction_end xs_single T xs_talkv read_reply event=7 itype=7 htype=7 len=NULL xs_read xs_single device/vbd/3/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL xenbus_write dir=device/vbd/3 val=3 xs_talkv read_reply event=11 itype=11 htype=11 len=NULL xs_talkv read_reply event=4 itype=4 htype=4 len=NULL xs_single device/vif xs_talkv read_reply event=1 itype=1 htype=1 len=2 gather xs_read xs_single /local/domain/0/backend/vbd/16/3/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/3/sectors xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 2048 xs_read xs_single /local/domain/0/backend/vbd/16/3/info xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 0 xs_read xs_single /local/domain/0/backend/vbd/16/3/sector-size xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 512 gather xs_read xs_single /local/domain/0/backend/vbd/16/3/feature-barrier xs_talkv read_reply event=16 itype=2 htype=16 len=NULL xs_read xs_single device/vbd/3/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL xenbus_write dir=device/vbd/3 val=4 xs_talkv read_reply event=11 itype=11 htype=11 len=NULL gather xs_read xs_single device/vif/0/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 1 gather xs_read xs_single device/vif/0/backend-id xs_talkv r event=2 handling watch itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/2/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/2/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/2/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/2/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/3/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/3/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/3/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/3/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 gather xs_read xs_single /local/domain/0/backend/vbd/16/1/state xs_talkv read_reply event=2 itype=2 htype=2 len=NULL 4 lock order reversal: (sleepable after non-sleepable) 1st 0xc17f701c struct mount mtx (struct mount mtx) @ /usr/src/sys/kern/vfs_subr.c:346 2nd 0xc17f7000 vfslock (vfslock) @ /usr/src/sys/kern/vfs_subr.c:373 KDB: stack backtrace: X_db_sym_numargs(c0343caf,c15e2b08,c0116f75,4,c033fa07,...) at X_db_sym_numargs+0x146 kdb_backtrace(4,c033fa07,c04e5930,c16476f0,c15e2b60,...) at kdb_backtrace+0x29 witness_display_spinlock(c0346572,c17f7000,c034cd9a,c16476f0,c034d33f,...) at witness_display_spinlock+0x75 witness_checkorder(c17f7000,1,c034d33f,175,c033fa07,...) at witness_checkorder+0x7c9 __lockmgr_args(c17f7000,200100,c17f701c,0,0,...) at __lockmgr_args+0x230 vfs_busy(c17f7000,200,0,c1684cc0,1,...) at vfs_busy+0x1bc vfs_mount_alloc(0,c037a0e0,c034d0e5,c1684cc0,c0156680,...) at vfs_mount_alloc+0x78 vfs_mountroot(c03afaf0,4,c033b8e6,264,0,...) at vfs_mountroot+0x26c exec_shell_imgact(0,c15e2d38,c033d306,322,c1682d0c,...) at exec_shell_imgact+0x6d5 fork_exit(c00a09e0,0,c15e2d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc15e2d70, ebp = 0 --- lock order reversal: (sleepable after non-sleepable) 1st 0xc170de10 vnode interlock (vnode interlock) @ /usr/src/sys/fs/devfs/devfs_vnops.c:286 2nd 0xc170ddf4 devfs (devfs) @ /usr/src/sys/kern/vfs_subr.c:2044 KDB: stack backtrace: X_db_sym_numargs(c0343caf,c15e2a2c,c0116f75,4,c033fa07,...) at X_db_sym_numargs+0x146 kdb_backtrace(4,c033fa07,c04e5930,c1647828,c15e2a84,...) at kdb_backtrace+0x29 witness_display_spinlock(c0346572,c170ddf4,c033761d,c1647828,c034d33f,...) at witness_display_spinlock+0x75 witness_checkorder(c170ddf4,9,c034d33f,7fc,155,...) at witness_checkorder+0x7c9 __lockmgr_args(c170ddf4,80100,c170de10,0,0,...) at __lockmgr_args+0x790 vop_stdlock(c15e2b88,c03377d3,c033aa54,80100,c170dd9c,...) at vop_stdlock+0x62 VOP_LOCK1_APV(c037a1c0,c15e2b88,c03aa020,c170dd9c,80100,...) at VOP_LOCK1_APV+0xa5 _vn_lock(c170dd9c,80100,c034d33f,7fc,8,...) at _vn_lock+0x5e vget(c170dd9c,80100,c1684cc0,11f,c0337770,...) at vget+0xc9 devfs_allocv(c17f3700,c17f7000,c15e2c20,c1684cc0,c1684d64,...) at devfs_allocv+0x11a devfs_rules_apply(c17f7000,80000,c0511714,c1684cc0,4,...) at devfs_rules_apply+0x151 vfs_mount_alloc(c0511700,0,c034cc9c,5f4,c0156680,...) at vfs_mount_alloc+0x15d vfs_mountroot(c03afaf0,4,c033b8e6,264,0,...) at vfs_mountroot+0x34c exec_shell_imgact(0,c15e2d38,c033d306,322,c1682d0c,...) at exec_shell_imgact+0x6d5 fork_exit(c00a09e0,0,c15e2d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xc15e2d70, ebp = 0 --- Trying to mount root from ufs:/dev/xbd0s1a Manual root filesystem specification: : Mount using filesystem eg. ufs:da0s1a ? List valid disk boot devices Abort manual input mountroot> bad stray interrupt panic: Root mount failed, startup aborted. cpuid = 0 KDB: enter: panic [thread pid 1 tid 100002 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> db> thanks again! -- Best regards, Brad