From owner-freebsd-questions@FreeBSD.ORG Tue Feb 26 18:30:18 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 973B35C2 for ; Tue, 26 Feb 2013 18:30:18 +0000 (UTC) (envelope-from bsd@todoo.biz) Received: from newmail.rmm.fr (newmail.rmm.fr [213.251.152.9]) by mx1.freebsd.org (Postfix) with ESMTP id F1CE81658 for ; Tue, 26 Feb 2013 18:30:17 +0000 (UTC) Received: from newmail.rmm.fr (newmail.rmm.fr [213.251.152.9]) by newmail.rmm.fr (Postfix) with ESMTP id 1A724153E58 for ; Tue, 26 Feb 2013 19:30:11 +0100 (CET) X-Virus-Scanned: amavisd-new 2.8.0 (20120630) at rmm.fr Received: from newmail.rmm.fr ([213.251.152.9]) by newmail.rmm.fr (newmail.rmm.fr [213.251.152.9]) (amavisd-new, port 10026) with ESMTP id Wy8fSFZfVDlg for ; Tue, 26 Feb 2013 19:30:10 +0100 (CET) Received: from newmail.rmm.fr (newmail.rmm.fr [87.98.206.99]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: hidden) by newmail.rmm.fr (Postfix) with ESMTPSA id D5018153E52 for ; Tue, 26 Feb 2013 19:30:10 +0100 (CET) From: "bsd@todoo.biz" Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: Hang on reboot with ZIL on SSD Message-Id: <6530A711-942C-4A16-94DD-572D67D7C9DC@todoo.biz> Date: Tue, 26 Feb 2013 19:30:09 +0100 To: Liste FreeBSD Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) X-Mailer: Apple Mail (2.1499) X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Feb 2013 18:30:18 -0000 Hi,=20 I have a quite big server that I am tuning with FreeNAS running on It.=20= It is based on an Intel Server and uses an Adaptec Controler ASR-6805 = for a potential 12 disks pool (only 6 deployed for the moment).=20 I have two more SSD disks intended for the ZIL cache. Connected directly = on the mother board.=20 It comes equipped with 32Gb of memory ECC.=20 The system is installed on a specific dongle on the mother board (4Gb = SLC dongle).=20 The system is happy (=3D reboots without stopping at the real end of the = reboot) as long as there is no SSD involved for the ZIL. As soon as the = SSD are running, system freezes (or at least can't proceed with the = reboot).=20 It really freezes at the real end of the reboot after :=20 > Syncing disks, vnodes remaining=85*0 0 0 0 done=20 > All buffers synced.=20 > Uptime: 3d4h12min I have to manually Power-Cycle the unit for It to complete the reboot.=20= Here is the output of the dmesg :=20 > [root@freenas] ~# dmesg=20 > Copyright (c) 1992-2012 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, = 1994 > The Regents of the University of California. All rights = reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 8.3-RELEASE-p5 #2 r244158M: Wed Dec 12 10:04:42 PST 2012 > = root@build.ixsystems.com:/home/jpaetzel/8.3.0/os-base/amd64/usr/home/jpaet= zel/8.3.0/FreeBSD/src/sys/FREENAS.amd64 amd64 > Timecounter "i8254" frequency 1193182 Hz quality 0 > CPU: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz (2394.25-MHz K8-class = CPU) > Origin =3D "GenuineIntel" Id =3D 0x206d7 Family =3D 6 Model =3D = 2d Stepping =3D 7 > = Features=3D0xbfebfbff > = Features2=3D0x17bee3ff > AMD Features=3D0x2c100800 > AMD Features2=3D0x1 > TSC: P-state invariant > real memory =3D 34359738368 (32768 MB) > avail memory =3D 33071357952 (31539 MB) > ACPI APIC Table: > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs > FreeBSD/SMP: 1 package(s) x 4 core(s) > cpu0 (BSP): APIC ID: 0 > cpu1 (AP): APIC ID: 2 > cpu2 (AP): APIC ID: 4 > cpu3 (AP): APIC ID: 6 > WARNING: VIMAGE (virtualized network stack) is a highly experimental = feature. > ACPI Warning: Invalid length for Pm1aControlBlock: 32, using default = 16 (20101013/tbfadt-707) > ioapic0 irqs 0-23 on motherboard > ioapic1 irqs 24-47 on motherboard > kbd1 at kbdmux0 > hpt27xx: RocketRAID 27xx controller driver v1.0 (Dec 12 2012 10:04:31) > cryptosoft0: on motherboard > aesni0: on motherboard > acpi0: on motherboard > acpi0: [ITHREAD] > acpi0: Power Button (fixed) > acpi0: reservation of 0, 9d000 (3) failed > Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 > acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 > cpu0: on acpi0 > cpu1: on acpi0 > cpu2: on acpi0 > cpu3: on acpi0 > pcib0: port 0xcf8-0xcff on acpi0 > pci0: on pcib0 > pcib1: irq 47 at device 1.0 on pci0 > pci1: on pcib1 > pcib2: irq 47 at device 1.1 on pci0 > pci2: on pcib2 > igb0: port = 0x1060-0x107f mem 0xd2160000-0xd217ffff,0xd21b0000-0xd21b3fff irq 27 at = device 0.0 on pci2 > igb0: Using MSIX interrupts with 5 vectors > igb0: Ethernet address: 00:1e:67:54:9f:cd > igb0: [ITHREAD] > igb0: [ITHREAD] > igb0: [ITHREAD] > igb0: [ITHREAD] > igb0: [ITHREAD] > igb1: port = 0x1040-0x105f mem 0xd2140000-0xd215ffff,0xd21a0000-0xd21a3fff irq 30 at = device 0.1 on pci2 > igb1: Using MSIX interrupts with 5 vectors > igb1: Ethernet address: 00:1e:67:54:9f:ce > igb1: [ITHREAD] > igb1: [ITHREAD] > igb1: [ITHREAD] > igb1: [ITHREAD] > igb1: [ITHREAD] > igb2: port = 0x1020-0x103f mem 0xd2120000-0xd213ffff,0xd2190000-0xd2193fff irq 28 at = device 0.2 on pci2 > igb2: Using MSIX interrupts with 5 vectors > igb2: Ethernet address: 00:1e:67:54:9f:cf > igb2: [ITHREAD] > igb2: [ITHREAD] > igb2: [ITHREAD] > igb2: [ITHREAD] > igb2: [ITHREAD] > igb3: port = 0x1000-0x101f mem 0xd2100000-0xd211ffff,0xd2180000-0xd2183fff irq 29 at = device 0.3 on pci2 > igb3: Using MSIX interrupts with 5 vectors > igb3: Ethernet address: 00:1e:67:54:9f:d0 > igb3: [ITHREAD] > igb3: [ITHREAD] > igb3: [ITHREAD] > igb3: [ITHREAD] > igb3: [ITHREAD] > pcib3: irq 47 at device 2.0 on pci0 > pci4: on pcib3 > pcib4: irq 47 at device 2.2 on pci0 > pci5: on pcib4 > pcib5: irq 16 at device 3.0 on pci0 > pci6: on pcib5 > aacu0: mem = 0xd1c00000-0xd1ffffff,0xd2050000-0xd20507ff,0xd2040000-0xd20400ff irq 40 = at device 0.0 on pci6 > aacu0: Enable Raw I/O > aacu0: Enable 64-bit array > aacu0: New comm. interface type1 enabled > aacu0: [ITHREAD] > aacu0: Adaptec 6805, aac driver 3.1.2-30035 > aacp0: on aacu0 > aacp1: on aacu0 > aacp2: on aacu0 > aacp3: on aacu0 > pcib6: irq 16 at device 3.2 on pci0 > pci7: on pcib6 > pci0: at device 4.0 (no driver attached) > pci0: at device 4.1 (no driver attached) > pci0: at device 4.2 (no driver attached) > pci0: at device 4.3 (no driver attached) > pci0: at device 4.4 (no driver attached) > pci0: at device 4.5 (no driver attached) > pci0: at device 4.6 (no driver attached) > pci0: at device 4.7 (no driver attached) > pci0: at device 5.0 (no driver attached) > pci0: at device 5.2 (no driver attached) > pcib7: irq 16 at device 17.0 on pci0 > pci8: on pcib7 > pci0: at device 22.0 (no driver attached) > pci0: at device 22.1 (no driver attached) > ehci0: mem 0xd2320000-0xd23203ff = irq 22 at device 26.0 on pci0 > ehci0: [ITHREAD] > usbus0: EHCI version 1.0 > usbus0: on ehci0 > pcib8: irq 16 at device 28.0 on pci0 > pci9: on pcib8 > pcib9: irq 19 at device 28.7 on pci0 > pci10: on pcib9 > vgapci0: mem = 0xd0000000-0xd0ffffff,0xd1810000-0xd1813fff,0xd1000000-0xd17fffff irq 19 = at device 0.0 on pci10 > ehci1: mem 0xd2310000-0xd23103ff = irq 20 at device 29.0 on pci0 > ehci1: [ITHREAD] > usbus1: EHCI version 1.0 > usbus1: on ehci1 > pcib10: at device 30.0 on pci0 > pci11: on pcib10 > isab0: at device 31.0 on pci0 > isa0: on isab0 > ahci0: port = 0x2070-0x2077,0x2060-0x2063,0x2050-0x2057,0x2040-0x2043,0x2020-0x203f = mem 0xd2300000-0xd23007ff irq 21 at device 31.2 on pci0 > ahci0: [ITHREAD] > ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported > ahcich0: at channel 0 on ahci0 > ahcich0: [ITHREAD] > ahcich1: at channel 1 on ahci0 > ahcich1: [ITHREAD] > pci0: at device 31.3 (no driver attached) > pcib11: on acpi0 > pci255: on pcib11 > pci255: at device 8.0 (no driver attached) > pci255: at device 9.0 (no driver attached) > pci255: at device 10.0 (no driver attached) > pci255: at device 10.1 (no driver attached) > pci255: at device 10.2 (no driver attached) > pci255: at device 10.3 (no driver attached) > pci255: at device 11.0 (no driver attached) > pci255: at device 11.3 (no driver attached) > pci255: at device 12.0 (no driver attached) > pci255: at device 12.1 (no driver attached) > pci255: at device 12.6 (no driver attached) > pci255: at device 12.7 (no driver attached) > pci255: at device 13.0 (no driver attached) > pci255: at device 13.1 (no driver attached) > pci255: at device 13.6 (no driver attached) > pci255: at device 14.0 (no driver attached) > pci255: at device 14.1 (no driver attached) > pci255: at device 15.0 (no driver attached) > pci255: at device 15.1 (no driver attached) > pci255: at device 15.2 (no driver attached) > pci255: at device 15.3 (no driver attached) > pci255: at device 15.4 (no driver attached) > pci255: at device 15.5 (no driver attached) > pci255: at device 15.6 (no driver attached) > pci255: at device 16.0 (no driver attached) > pci255: at device 16.1 (no driver attached) > pci255: at device 16.2 (no driver attached) > pci255: at device 16.3 (no driver attached) > pci255: at device 16.4 (no driver attached) > pci255: at device 16.5 (no driver attached) > pci255: at device 16.6 (no driver attached) > pci255: at device 16.7 (no driver attached) > pci255: at device 17.0 (no driver attached) > pci255: at device 19.0 (no driver attached) > pci255: at device 19.1 (no driver attached) > pci255: at device 19.4 (no driver attached) > pci255: at device 19.5 (no driver attached) > pci255: at device 19.6 (no driver attached) > acpi_hpet0: iomem 0xfed00000-0xfed003ff = on acpi0 > Timecounter "HPET" frequency 14318180 Hz quality 900 > atrtc0: port 0x70-0x77 irq 8 on acpi0 > atrtc0: Warning: Couldn't map I/O. > uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on = acpi0 > uart0: [FILTER] > uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 > uart1: [FILTER] > orm0: at iomem = 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0xc9fff,0xca000-0xcafff,0xcb000-0x= cbfff,0xcc000-0xd27ff on isa0 > sc0: at flags 0x100 on isa0 > sc0: VGA <16 virtual consoles, flags=3D0x300> > vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on = isa0 > atkbdc0: at port 0x60,0x64 on isa0 > atkbd0: irq 1 on atkbdc0 > kbd0 at atkbd0 > atkbd0: [GIANT-LOCKED] > atkbd0: [ITHREAD] > ppc0: cannot reserve I/O port range > coretemp0: on cpu0 > est0: on cpu0 > p4tcc0: on cpu0 > coretemp1: on cpu1 > est1: on cpu1 > est: CPU supports Enhanced Speedstep, but is not recognized. > est: cpu_vendor GenuineIntel, msr 1ee700001800 > device_attach: est1 attach returned 6 > p4tcc1: on cpu1 > coretemp2: on cpu2 > est2: on cpu2 > p4tcc2: on cpu2 > coretemp3: on cpu3 > est3: on cpu3 > est: CPU supports Enhanced Speedstep, but is not recognized. > est: cpu_vendor GenuineIntel, msr 1f1000001800 > device_attach: est3 attach returned 6 > p4tcc3: on cpu3 > fuse4bsd: version 0.3.9-pre1, FUSE ABI 7.8 > Timecounters tick every 1.000 msec > hpt27xx: no controller detected. > usbus0: 480Mbps High Speed USB v2.0 > usbus1: 480Mbps High Speed USB v2.0 > ugen0.1: at usbus0 > uhub0: on = usbus0 > ugen1.1: at usbus1 > uhub1: on = usbus1 > uhub0: 2 ports with 2 removable, self powered > uhub1: 2 ports with 2 removable, self powered > ugen0.2: at usbus0 > uhub2: on usbus0 > ugen1.2: at usbus1 > uhub3: on usbus1 > uhub2: 6 ports with 6 removable, self powered > uhub3: 8 ports with 8 removable, self powered > ugen1.3: at usbus1 > umass0: on usbus1 > ugen1.4: at usbus1 > ukbd0: = on usbus1 > kbd2 at ukbd0 > uhid0: = on usbus1 > ugen1.5: at usbus1 > ukbd1: on usbus1 > kbd3 at ukbd1 > ums0: on usbus1 > ums0: 3 buttons and [Z] coordinates ID=3D0 > ada0 at ahcich0 bus 0 scbus4 target 0 lun 0 > ada0: ATA-8 SATA 2.x device > ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > ada0: Command Queueing enabled > ada0: 95396MB (195371568 512 byte sectors: 16H 63S/T 16383C) > ada1 at ahcich1 bus 0 scbus5 target 0 lun 0 > ada1: ATA-8 SATA 2.x device > ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > ada1: Command Queueing enabled > ada1: 95396MB (195371568 512 byte sectors: 16H 63S/T 16383C) > da6 at umass-sim0 bus 0 scbus6 target 0 lun 0 > da6: Fixed Direct Access SCSI-0 device=20 > da6: 40.000MB/s transfers > da6: 3920MB (8028160 512 byte sectors: 255H 63S/T 499C) > SMP: AP CPU #1 Launched! > SMP: AP CPU #2 Launched! > SMP: AP CPU #3 Launched! > da0 at aacp1 bus 0 scbus1 target 0 lun 0 > da0: Fixed Direct Access SCSI-5 device=20 > da0: 300.000MB/s transfers > da0: Command Queueing enabled > da0: 2856950MB (5851033600 512 byte sectors: 255H 63S/T 364209C) > da1 at aacp1 bus 0 scbus1 target 1 lun 0 > da1: Fixed Direct Access SCSI-5 device=20 > da1: 300.000MB/s transfers > da1: Command Queueing enabled > da1: 2856950MB (5851033600 512 byte sectors: 255H 63S/T 364209C) > da2 at aacp1 bus 0 scbus1 target 4 lun 0 > da2: Fixed Direct Access SCSI-5 device=20 > da2: 300.000MB/s transfers > da2: Command Queueing enabled > da2: 2856950MB (5851033600 512 byte sectors: 255H 63S/T 364209C) > da3 at aacp1 bus 0 scbus1 target 5 lun 0 > da3: Fixed Direct Access SCSI-5 device=20 > da3: 300.000MB/s transfers > da3: Command Queueing enabled > da3: 2856950MB (5851033600 512 byte sectors: 255H 63S/T 364209C) > da4 at aacp1 bus 0 scbus1 target 6 lun 0 > da4: Fixed Direct Access SCSI-5 device=20 > da4: 300.000MB/s transfers > da4: Command Queueing enabled > da4: 2856950MB (5851033600 512 byte sectors: 255H 63S/T 364209C) > da5 at aacp1 bus 0 scbus1 target 7 lun 0 > da5: Fixed Direct Access SCSI-5 device=20 > da5: 300.000MB/s transfers > da5: Command Queueing enabled > da5: 2856950MB (5851033600 512 byte sectors: 255H 63S/T 364209C) > GEOM: da6s1: geometry does not match label (16h,63s !=3D 255h,63s). > Trying to mount root from ufs:/dev/ufs/FreeNASs1a > ZFS filesystem version 5 > ZFS storage pool version 28 Here is an example of the pool with only one disk used for ZIL (not the = target) - But I tried this in order to see how It worked=85=20 > [root@freenas] ~# zpool status > pool: tank > state: ONLINE > scan: none requested > config: >=20 > NAME STATE READ = WRITE CKSUM > tank ONLINE 0 = 0 0 > raidz2-0 ONLINE 0 = 0 0 > gptid/053bf90c-803f-11e2-a8db-001e67549fcd ONLINE 0 = 0 0 > gptid/08a7dcbd-803f-11e2-a8db-001e67549fcd ONLINE 0 = 0 0 > gptid/0c0767d0-803f-11e2-a8db-001e67549fcd ONLINE 0 = 0 0 > gptid/0f6ac358-803f-11e2-a8db-001e67549fcd ONLINE 0 = 0 0 > gptid/12ce2b68-803f-11e2-a8db-001e67549fcd ONLINE 0 = 0 0 > gptid/162f71c0-803f-11e2-a8db-001e67549fcd ONLINE 0 = 0 0 > logs > gptid/17980c47-803f-11e2-a8db-001e67549fcd ONLINE 0 = 0 0 >=20 > errors: No known data errors Is there some tuning to be done on the Bios when you use disks as ZIL ?=20= Thanks for your feedback.=20 G.B.=20 ________________________________________________ =AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?= =BB=A7=AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?=BB=A7 =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF= =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF Your provider of OpenSource Appliances www.osnet.eu =09 ________________________________________________ =AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?= =BB=A7=AB?=BB=A5=AB?=BB=A7=AB?=BB=A5=AB?=BB=A7 =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF= =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF PGP ID --> 0x1BA3C2FD