From owner-freebsd-geom@FreeBSD.ORG Tue Apr 3 02:34:33 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 11F2716A405 for ; Tue, 3 Apr 2007 02:34:33 +0000 (UTC) (envelope-from anton.p3@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.248]) by mx1.freebsd.org (Postfix) with ESMTP id D2C4D13C484 for ; Tue, 3 Apr 2007 02:34:31 +0000 (UTC) (envelope-from anton.p3@gmail.com) Received: by an-out-0708.google.com with SMTP id c24so1514786ana for ; Mon, 02 Apr 2007 19:34:31 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type; b=q7cs17ioO9ROPYcG+eC/CBoT6Mvq03OnZIL3TWmp9goQwORp8quTN392DZNx+NLsWdHikyOLwxZ28pRLFWG+igx395u4uQJl8R1DGD2wBgvW4ffO3vgbUT1W1Zj1mbEbwP0l4rFN9LXK1mlz6IhILK6zN7bPJJnAIZuytRdrg/w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:mime-version:content-type; b=IYrhDeZ18KvlJKjzEAMKlEyXEX3FlaWIp6K8JNFrvfM0ParJRNlH0u0L6oSh0Edat/NBXSGME8pcml2C98KjgUfxliZe7aLTPeXfHlaN8Wz0MhL1ivMp89CIyTDXzPyEn1vvTbTsr03MGmPy2zCcqSQDiI3bGOMyA0LmZsRdEAs= Received: by 10.100.111.16 with SMTP id j16mr3977916anc.1175566218219; Mon, 02 Apr 2007 19:10:18 -0700 (PDT) Received: by 10.100.122.19 with HTTP; Mon, 2 Apr 2007 19:10:17 -0700 (PDT) Message-ID: Date: Mon, 2 Apr 2007 22:10:17 -0400 From: "A P" To: freebsd-geom@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: File corruption using gmirror via Sil3114 X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Apr 2007 02:34:33 -0000 Hi, I have the motherboard with PCI SATA RAID card (Sil3114) that I am using only to utilize SATA interface. I was hoping that gmirror provides more tools and more reliable. I setup the mirror using gmirror on 2 identical drives. When gmirror is in a complete state (not degraded) I can observe a file corruption: ~ > uname -a FreeBSD server 6.2-RELEASE FreeBSD 6.2-RELEASE #0: Fri Jan 12 10:40:27 UTC 2007 root@dessler.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 ~ > cp /d2/qemu/backup/winxp.img /d2/qemu/winxp00.img ~ > md5 /d2/qemu/winxp00.img MD5 (/d2/qemu/winxp00.img) = 8b74f5796d84b93412964fdb89fd4203 ~ > diff /d2/qemu/backup/winxp.img /d2/qemu/winxp00.img Binary files /d2/qemu/backup/winxp.img and /d2/qemu/winxp00.img differ ~ > md5 /d2/qemu/backup/winxp.img MD5 (/d2/qemu/backup/winxp.img) = 9cfc1ea88d1099e59352c7a4b6a670bd ~ > md5 /d2/qemu/backup/winxp.img MD5 (/d2/qemu/backup/winxp.img) = 9cfc1ea88d1099e59352c7a4b6a670bd ~ > md5 /d2/qemu/winxp00.img MD5 (/d2/qemu/winxp00.img) = be177f05a63dd52b26cf8323d33140e8 ~ > sync ~ > md5 /d2/qemu/winxp00.img MD5 (/d2/qemu/winxp00.img) = be177f05a63dd52b26cf8323d33140e8 I see no errors on the console, /var/log/all.log or /var/log/messages. I have tested memory with the memtest and was not able to find any errors during 12 hours of constant testing. Gmirror configuration: ~ > gmirror list geom name: gm0 State: COMPLETE Components: 2 Balance: split Slice: 4096 Flags: NONE GenID: 1 SyncID: 5 ID: 514065193 Providers: 1. Name: mirror/gm0 Mediasize: 250059349504 (233G) Sectorsize: 512 Mode: r3w3e4 Consumers: 1. Name: ad4 Mediasize: 250059350016 (233G) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Priority: 0 Flags: NONE GenID: 1 SyncID: 5 ID: 463706396 2. Name: ad8 Mediasize: 250059350016 (233G) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Priority: 0 Flags: NONE GenID: 1 SyncID: 5 ID: 2907353337 ~> gmirror status Name Status Components mirror/gm0 COMPLETE ad4 ad8 Now the interesting part. If I remove any (I tested both) drive from the mirror the problem goes away. anton ~ > gmirror list Geom name: gm0 State: DEGRADED Components: 2 Balance: split Slice: 4096 Flags: NONE GenID: 1 SyncID: 9 ID: 514065193 Providers: 1. Name: mirror/gm0 Mediasize: 250059349504 (233G) Sectorsize: 512 Mode: r3w3e4 Consumers: 1. Name: ad8 Mediasize: 250059350016 (233G) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Priority: 0 Flags: NONE GenID: 1 SyncID: 9 ID: 2907353337 anton ~ > gmirror status Name Status Components mirror/gm0 DEGRADED ad8 ~ > cp /d2/qemu/backup/winxp.img /d2/qemu/winxp00.img ~ > diff /d2/qemu/backup/winxp.img /d2/qemu/winxp00.img ~ > ls -l /d2/qemu/backup/winxp.img -rw-r--r-- 1 anton wheel 1282801664 Mar 5 11:11 /d2/qemu/backup/winxp.img ~ > df -t ufs -h Filesystem Size Used Avail Capacity Mounted on /dev/mirror/gm0s1a 12G 2.3G 8.4G 21% / /dev/mirror/gm0s1d 208G 8.8G 183G 5% /d2 As soon as I activate the gmirror and wait until rebuilding is finished the problem comes back. ~ > dmesg Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 6.2-RELEASE #0: Fri Jan 12 10:40:27 UTC 2007 root@dessler.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC WARNING: MPSAFE network stack disabled, expect reduced performance. ACPI APIC Table: Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: AMD Athlon(tm) XP 2600+ (1908.95-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x6a0 Stepping = 0 Features=0x383fbff AMD Features=0xc0400800 real memory = 2130640896 (2031 MB) avail memory = 2079977472 (1983 MB) ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 ath_hal: 0.9.17.2 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 cpu0: on acpi0 acpi_button0: on acpi0 acpi_button1: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 agp0: mem 0xe7000000-0xe73fffff at device 0.0 on pci0 pcib1: at device 1.0 on pci0 pci1: on pcib1 pci1: at device 0.0 (no driver attached) atapci0: port 0xc000-0xc007,0xc400-0xc403,0xc800-0xc807,0xcc00-0xcc03,0xd000-0xd00f mem 0xe7410000-0xe74103ff irq 16 at device 8.0 on pci0 ata2: on atapci0 ata3: on atapci0 ata4: on atapci0 ata5: on atapci0 bge0: mem 0xe7400000-0xe740ffff irq 17 at device 9.0 on pci0 miibus0: on bge0 brgphy0: on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto bge0: Ethernet address: 00:04:76:e8:77:cb bge0: [GIANT-LOCKED] uhci0: port 0xd400-0xd41f irq 21 at device 16.0on pci0 uhci0: [GIANT-LOCKED] usb0: on uhci0 usb0: USB revision 1.0 uhub0: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xd800-0xd81f irq 21 at device 16.1on pci0 uhci1: [GIANT-LOCKED] usb1: on uhci1 usb1: USB revision 1.0 uhub1: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xdc00-0xdc1f irq 21 at device 16.2on pci0 uhci2: [GIANT-LOCKED] usb2: on uhci2 usb2: USB revision 1.0 uhub2: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub2: 2 ports with 2 removable, self powered ehci0: mem 0xe7411000-0xe74110ff irq 21 at device 16.3 on pci0 ehci0: [GIANT-LOCKED] usb3: EHCI version 1.0 usb3: companion controllers, 2 ports each: usb0 usb1 usb2 usb3: on ehci0 usb3: USB revision 2.0 uhub3: VIA EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 uhub3: 6 ports with 6 removable, self powered isab0: at device 17.0 on pci0 isa0: on isab0 atapci1: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe000-0xe00f at device 17.1 on pci0 ata0: on atapci1 ata1: on atapci1 vr0: port 0xe800-0xe8ff mem 0xe7412000-0xe74120ff irq 23 at device 18.0 on pci0 miibus1: on vr0 ukphy0: on miibus1 ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto vr0: Ethernet address: 00:0e:a6:c5:09:7a vr0: [GIANT-LOCKED] acpi_tz0: on acpi0 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model Generic PS/2 mouse, device ID 0 pmtimer0 on isa0 orm0: at iomem 0xd0000-0xd47ff on isa0 ppc0: parallel port not found. sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 1908953027 Hz quality 800 Timecounters tick every 1.000 msec acd0: CDROM at ata0-master UDMA33 ad4: 238475MB at ata2-master SATA150 GEOM_MIRROR: Device gm0 created (id=514065193). GEOM_MIRROR: Device gm0: provider ad4 detected. ad8: 238475MB at ata4-master SATA150 GEOM_MIRROR: Device gm0: provider ad8 detected. GEOM_MIRROR: Device gm0: provider ad8 activated. GEOM_MIRROR: Device gm0: provider ad4 activated. GEOM_MIRROR: Device gm0: provider mirror/gm0 launched. Trying to mount root from ufs:/dev/mirror/gm0s1a bge0: link state changed to UP kqemu version 0x00010300 kqemu: KQEMU installed, max_locked_mem=1035984kB. Am I doing something wrong? Let me know if any other additional info needed, Thank you, Anton