From owner-freebsd-current Tue Mar 21 11:59:50 2000 Delivered-To: freebsd-current@freebsd.org Received: from trinity.skynet.be (trinity.skynet.be [195.238.2.38]) by hub.freebsd.org (Postfix) with ESMTP id 39F8A37C1A9 for ; Tue, 21 Mar 2000 11:59:40 -0800 (PST) (envelope-from blk@skynet.be) Received: from [195.238.1.121] (brad.techos.skynet.be [195.238.1.121]) by trinity.skynet.be (Postfix) with ESMTP id 7AE4D182CB; Tue, 21 Mar 2000 20:57:06 +0100 (MET) Mime-Version: 1.0 X-Sender: blk@pop.skynet.be Message-Id: Date: Tue, 21 Mar 2000 20:53:24 +0100 To: Greg Lehey From: Brad Knowles Subject: Problems with vinum and/or rawio? Cc: FreeBSD-CURRENT Mailing List Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Greg, Running 4.0-STABLE (cvsup'ed a couple of days ago), I'm having a problem with a vinum striped volume that I've set up. The machine is a dual-CPU Pentium III/450 (Dell 1300) with 1GB of RAM and 512KB L2 cache per processor, on an IBM DDRS-39130D DC2 8GB hard drive. The external drive array is a Comparex D1400 (Hitachi DF400), with four separate logical units (one for each row of disks), and two logical units exported exclusively through one interface on one controller, which is attached exclusively to one Adaptec 2940U2W host adaptor. The IBM drive with the OS, etc... is attached to the on-board Adaptec AIC-7980 controller chip. The command I had run was: rawio -c 128 -p 16 -n 65536 -r -R /dev/vinum/news I got part way through the output being generated, when I started getting a lot of errors like this: Mar 21 20:48:46 audrey /kernel: (da4:ahc1:0:6:0): SCB 0x20 - timed out in Data-in phase, SEQADDR == 0x5d Mar 21 20:48:46 audrey /kernel: (da4:ahc1:0:6:0): SCB 0x20 - timed out in Data-in phase, SEQADDR == 0x5d Mar 21 20:48:46 audrey /kernel: (da4:ahc1:0:6:0): BDR message in message buffer Mar 21 20:48:46 audrey /kernel: (da4:ahc1:0:6:0): BDR message in message buffer Mar 21 20:48:48 audrey /kernel: (da4:ahc1:0:6:0): SCB 0x20 - timed out in Data-in phase, SEQADDR == 0x5d Mar 21 20:48:48 audrey /kernel: (da4:ahc1:0:6:0): SCB 0x20 - timed out in Data-in phase, SEQADDR == 0x5d Mar 21 20:48:48 audrey /kernel: (da4:ahc1:0:6:0): no longer in timeout, status = 34b Mar 21 20:48:48 audrey /kernel: (da4:ahc1:0:6:0): no longer in timeout, status = 34b Mar 21 20:48:48 audrey /kernel: ahc1: Issued Channel A Bus Reset. 11 SCBs aborted Mar 21 20:48:48 audrey /kernel: ahc1: Issued Channel A Bus Reset. 11 SCBs aborted Once this happened, all of the rawio processes were in "physst", but the system had 100% idle time. Do you need to see my kernel configuration? Beyond the vinum configuration and the output of dmesg, is there any other configuration details you need? Thanks! My /etc/vinum.conf: drive d1 device /dev/da2s1e drive d2 device /dev/da3s1e drive d3 device /dev/da4s1e drive d4 device /dev/da5s1e volume news plex org striped 512k sd length 0 drive d1 sd length 0 drive d2 sd length 0 drive d3 sd length 0 drive d4 dmesg: $ dmesg Copyright (c) 1992-2000 The FreeBSD Project. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 4.0-STABLE #0: Mon Mar 20 21:06:56 CET 2000 root@audrey.skynet.be:/usr/src/sys/compile/AUDREY Timecounter "i8254" frequency 1193182 Hz CPU: Pentium III/Pentium III Xeon (448.62-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x673 Stepping = 3 Features=0x383fbff real memory = 1073741824 (1048576K bytes) config> di sn0 No such device: sn0 Invalid command or syntax. Type `?' for help. config> di lnc0 No such device: lnc0 Invalid command or syntax. Type `?' for help. config> di le0 No such device: le0 Invalid command or syntax. Type `?' for help. config> di ie0 No such device: ie0 Invalid command or syntax. Type `?' for help. config> di fe0 No such device: fe0 Invalid command or syntax. Type `?' for help. config> di ed0 No such device: ed0 Invalid command or syntax. Type `?' for help. config> di cs0 No such device: cs0 Invalid command or syntax. Type `?' for help. config> q avail memory = 1038483456 (1014144K bytes) Programming 24 pins in IOAPIC #0 IOAPIC #0 intpin 2 -> irq 0 FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 1, version: 0x00040011, at 0xfee00000 cpu1 (AP): apic id: 0, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 2, version: 0x00170011, at 0xfec00000 Preloaded elf kernel "kernel" at 0xc0399000. Preloaded userconfig_script "/boot/kernel.conf" at 0xc039909c. ccd0-3: Concatenated disk drivers Pentium Pro MTRR support enabled md1: Malloc disk npx0: on motherboard npx0: INT 16 interface pcib0: on motherboard pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 pci1: at 0.0 pcib2: at device 2.0 on pci0 pci2: on pcib2 ahc0: port 0xdc00-0xdcff mem 0xf9fff000-0xf9ffffff irq 21 at device 9.0 on pci2 ahc0: aic7890/91 Wide Channel A, SCSI Id=7, 16/255 SCBs ahc1: port 0xd800-0xd8ff mem 0xf9ffe000-0xf9ffefff irq 22 at device 10.0 on pci2 ahc1: aic7890/91 Wide Channel A, SCSI Id=7, 16/255 SCBs ahc2: port 0xd400-0xd4ff mem 0xf9ffd000-0xf9ffdfff irq 16 at device 11.0 on pci2 ahc2: aic7890/91 Wide Channel A, SCSI Id=7, 16/255 SCBs isab0: at device 7.0 on pci0 isa0: on isab0 atapci0: port 0xffa0-0xffaf at device 7.1 on pci0 ata1: at 0x170 irq 15 on atapci0 pci0: at 7.2 irq 19 Timecounter "PIIX" frequency 3579545 Hz intpm0: port 0x850-0x85f irq 9 at device 7.3 on pci0 intpm0: I/O mapped 850 intpm0: intr IRQ 9 enabled revision 0 smbus0: on intsmb0 smb0: on smbus0 intpm0: PM I/O mapped 800 fxp0: port 0xccc0-0xccdf mem 0xfe000000-0xfe0fffff,0xf7000000-0xf7000fff irq 19 at device 16.0 on pci0 fxp0: Ethernet address 00:90:27:99:13:1a fxp0: supplying EUI64: 00:90:27:ff:fe:99:13:1a vt0 on isa0 vt0: generic, 80 col, color, 9 scr, unknown kbd, [R3.20-b24] vt0: driver is using old-style compatability shims atkbdc0: at port 0x60-0x6f on isa0 atkbd0: irq 1 on atkbdc0 vga0: at port 0x3b0-0x3df iomem 0xa0000-0xbffff on isa0 sc0: on isa0 sc0: VGA <16 virtual consoles, flags=0x200> fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fdc0: FIFO enabled, 8 bytes threshold fd0: <1440-KB 3.5" drive> on fdc0 drive 0 sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A esp_port has com 3 esp_port has com 3 esp_port has com 3 pcf0: at port 0x320-0x321 irq 5 on isa0 iicbus0: on pcf0 addr 0xaa iicsmb0: on iicbus0 smbus1: on iicsmb0 smb1: on smbus1 iic0: on iicbus0 ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/8 bytes threshold APIC_IO: Testing 8254 interrupt delivery APIC_IO: routing 8254 via IOAPIC #0 intpin 2 IP packet filtering initialized, divert enabled, rule-based forwarding enabled, default to accept, logging limited to 100 packets/entry by default DUMMYNET initialized (000106) BRIDGE 990810, have 9 interfaces -- index 1 type 6 phy 0 addrl 6 addr 00.90.27.99.13.1a IPsec: Initialized Security Association Processing. IPv6 packet filtering initialized, default to accept, logging limited to 100 packets/entry SMP: AP CPU #1 Launched! Waiting 5 seconds for SCSI devices to settle da2 at ahc0 bus 0 target 5 lun 0 da2: Fixed Direct Access SCSI-2 device da2: 80.000MB/s transfers (40.000MHz, offset 15, 16bit), Tagged Queueing Enabled da2: 68235MB (139745280 512 byte sectors: 255H 63S/T 8698C) da4 at ahc1 bus 0 target 6 lun 0 da4: Fixed Direct Access SCSI-2 device da4: 80.000MB/s transfers (40.000MHz, offset 15, 16bit), Tagged Queueing Enabled da4: 68235MB (139745280 512 byte sectors: 255H 63S/T 8698C) da0 at ahc2 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-3 device da0: 80.000MB/s transfers (40.000MHz, offset 31, 16bit), Tagged Queueing Enabled da0: 8683MB (17783249 512 byte sectors: 255H 63S/T 1106C) da3 at ahc0 bus 0 target 5 lun 1 da3: Fixed Direct Access SCSI-2 device da3: 80.000MB/s transfers (40.000MHz, offset 15, 16bit), Tagged Queueing Enabled da3: 68235MB (139745280 512 byte sectors: 255H 63S/T 8698C) da5 at ahc1 bus 0 target 6 lun 1 da5: Fixed Direct Access SCSI-2 device da5: 80.000MB/s transfers (40.000MHz, offset 15, 16bit), Tagged Queueing Enabled da5: 68235MB (139745280 512 byte sectors: 255H 63S/T 8698C) Mounting root from ufs:/dev/da1s1a da1 at ahc2 bus 0 target 1 lun 0 da1: Fixed Direct Access SCSI-2 device da1: 80.000MB/s transfers (40.000MHz, offset 15, 16bit), Tagged Queueing Enabled da1: 8715MB (17850000 512 byte sectors: 255H 63S/T 1111C) fxp0: starting DAD for fe80:0001::0290:27ff:fe99:131a fxp0: DAD complete for fe80:0001::0290:27ff:fe99:131a - no duplicates found vinum: loaded vinum: drive d1 is up vinum: drive d2 is up vinum: drive d3 is up vinum: drive d4 is up vinum: removing 1840 blocks of partial stripe at the end of news.p0 vinum: news.p0.s0 is up vinum: news.p0.s1 is up vinum: news.p0.s2 is up vinum: news.p0.s3 is up vinum: news.p0 is up vinum: news is up (da4:ahc1:0:6:0): SCB 0x36 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x36 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 16 SCBs aborted (da4:ahc1:0:6:0): SCB 0x36 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x36 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 16 SCBs aborted (da4:ahc1:0:6:0): SCB 0x36 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x36 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 16 SCBs aborted (da4:ahc1:0:6:0): SCB 0x36 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x36 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 16 SCBs aborted (da4:ahc1:0:6:0): SCB 0x36 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x36 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 16 SCBs aborted news.p0.s2: fatal read I/O error vinum: news.p0.s2 is crashed by force vinum: news.p0 is corrupt (da4:ahc1:0:6:0): SCB 0x29 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x29 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 15 SCBs aborted (da4:ahc1:0:6:0): SCB 0x29 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x29 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 15 SCBs aborted (da4:ahc1:0:6:0): SCB 0x29 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x29 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 15 SCBs aborted (da4:ahc1:0:6:0): SCB 0x29 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x29 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 15 SCBs aborted (da4:ahc1:0:6:0): SCB 0x29 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x29 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 15 SCBs aborted news.p0.s2: fatal read I/O error (da4:ahc1:0:6:0): SCB 0x16 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x16 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 13 SCBs aborted (da4:ahc1:0:6:0): SCB 0x16 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x16 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 13 SCBs aborted (da4:ahc1:0:6:0): SCB 0x16 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x16 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 13 SCBs aborted (da4:ahc1:0:6:0): SCB 0x16 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x16 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 13 SCBs aborted (da4:ahc1:0:6:0): SCB 0x16 - timed out in Data-in phase, SEQADDR == 0x5e (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x16 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 13 SCBs aborted news.p0.s2: fatal read I/O error (da4:ahc1:0:6:0): SCB 0x20 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): BDR message in message buffer (da4:ahc1:0:6:0): SCB 0x20 - timed out in Data-in phase, SEQADDR == 0x5d (da4:ahc1:0:6:0): no longer in timeout, status = 34b ahc1: Issued Channel A Bus Reset. 11 SCBs aborted -- These are my opinions -- not to be taken as official Skynet policy ====================================================================== Brad Knowles, || Belgacom Skynet SA/NV Systems Architect, Mail/News/FTP/Proxy Admin || Rue Colonel Bourg, 124 Phone/Fax: +32-2-706.13.11/12.49 || B-1140 Brussels http://www.skynet.be || Belgium To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message