Date: Thu, 6 Nov 2014 01:27:39 +0100 From: Kai Gallasch <k@free.de> To: freebsd-stable@freebsd.org Subject: Re: 10.1 RC4 r273903 - zpool scrub on ssd mirror - ahci command timeout Message-ID: <20141106012739.509b96b5@orwell> In-Reply-To: <545AB64F.1060502@multiplay.co.uk> References: <20141106003240.344dedf6@orwell> <545AB64F.1060502@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
--Sig_/H3gB3vpPwbhWohn56fzs7vY Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Am Wed, 05 Nov 2014 23:44:15 +0000 schrieb Steven Hartland <killing@multiplay.co.uk>: > Looks like a HW issue, how is it connected to the controller and what > is the controller? It's a SUN X2270 M1 server with 4 SATA connected disks. The (2.5") SSD drives are inside SUN 3.5" disk carriers and are sitting on top of a 2.5" to 3.5" converter - like this one here http://goo.gl/yBfS1V dmesg.out: Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.1-RC4 #0 r273903: Fri Oct 31 15:32:22 CET 2014 kgal@beltwaybandit.free.de:/usr/obj/usr/src/sys/X2270-10 amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 CPU: Intel(R) Xeon(R) CPU L5520 @ 2.27GHz (2266.80-MHz K8-class = CPU) Origin =3D "GenuineIntel" Id =3D 0x106a5 Family =3D 0x6 Model =3D 0x1a= Stepping =3D 5 Features=3D0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PG= E,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=3D0x9ce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,P= DCM,DCA,SSE4.1,SSE4.2,POPCNT> AMD Features=3D0x28100800<SYSCALL,NX,RDTSCP,LM> AMD Features2=3D0x1<LAHF> VT-x: PAT,HLT,MTF,PAUSE,EPT,VPID TSC: P-state invariant, performance statistics real memory =3D 51539607552 (49152 MB) avail memory =3D 50000900096 (47684 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: <040910 APIC1030> FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 SMT threads cpu0 (BSP): APIC ID: 16 cpu1 (AP): APIC ID: 17 cpu2 (AP): APIC ID: 18 cpu3 (AP): APIC ID: 19 cpu4 (AP): APIC ID: 20 cpu5 (AP): APIC ID: 21 cpu6 (AP): APIC ID: 22 cpu7 (AP): APIC ID: 23 cpu8 (AP): APIC ID: 0 cpu9 (AP): APIC ID: 1 cpu10 (AP): APIC ID: 2 cpu11 (AP): APIC ID: 3 cpu12 (AP): APIC ID: 4 cpu13 (AP): APIC ID: 5 cpu14 (AP): APIC ID: 6 cpu15 (AP): APIC ID: 7 ioapic0 <Version 2.0> irqs 0-23 on motherboard ioapic1 <Version 2.0> irqs 24-47 on motherboard random: <Software, Yarrow> initialized module_register_init: MOD_LOAD (vesa, 0xffffffff8085c590, 0) error 19 kbd1 at kbdmux0 cryptosoft0: <software crypto> on motherboard acpi0: <040910 XSDT1030> on motherboard acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, bff00000 (3) failed cpu0: <ACPI CPU> on acpi0 cpu1: <ACPI CPU> on acpi0 cpu2: <ACPI CPU> on acpi0 cpu3: <ACPI CPU> on acpi0 cpu4: <ACPI CPU> on acpi0 cpu5: <ACPI CPU> on acpi0 cpu6: <ACPI CPU> on acpi0 cpu7: <ACPI CPU> on acpi0 cpu8: <ACPI CPU> on acpi0 cpu9: <ACPI CPU> on acpi0 cpu10: <ACPI CPU> on acpi0 cpu11: <ACPI CPU> on acpi0 cpu12: <ACPI CPU> on acpi0 cpu13: <ACPI CPU> on acpi0 cpu14: <ACPI CPU> on acpi0 cpu15: <ACPI CPU> on acpi0 attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0 pci4: <ACPI PCI bus> on pcib1 igb0: <Intel(R) PRO/1000 Network Connection version - 2.4.0> port 0xec00-0x= ec1f mem 0xfbee0000-0xfbefffff,0xfbec0000-0xfbedffff,0xfbebc000-0xfbebffff = irq 40 at device 0.0 on pci4 igb0: Using MSIX interrupts with 5 vectors igb0: Ethernet address: 00:14:4f:ca:a9:00 igb0: Bound queue 0 to cpu 0 igb0: Bound queue 1 to cpu 1 igb0: Bound queue 2 to cpu 2 igb0: Bound queue 3 to cpu 3 igb1: <Intel(R) PRO/1000 Network Connection version - 2.4.0> port 0xe880-0x= e89f mem 0xfbe60000-0xfbe7ffff,0xfbe40000-0xfbe5ffff,0xfbe3c000-0xfbe3ffff = irq 28 at device 0.1 on pci4 igb1: Using MSIX interrupts with 5 vectors igb1: Ethernet address: 00:14:4f:ca:a9:01 igb1: Bound queue 0 to cpu 4 igb1: Bound queue 1 to cpu 5 igb1: Bound queue 2 to cpu 6 igb1: Bound queue 3 to cpu 7 pcib2: <ACPI PCI-PCI bridge> at device 3.0 on pci0 pci3: <ACPI PCI bus> on pcib2 pcib3: <ACPI PCI-PCI bridge> at device 7.0 on pci0 pci2: <ACPI PCI bus> on pcib3 pci0: <base peripheral, interrupt controller> at device 20.0 (no driver att= ached) pci0: <base peripheral, interrupt controller> at device 20.1 (no driver att= ached) pci0: <base peripheral, interrupt controller> at device 20.2 (no driver att= ached) pci0: <base peripheral, interrupt controller> at device 20.3 (no driver att= ached) uhci0: <Intel 82801JI (ICH10) USB controller USB-D> port 0xb800-0xb81f irq = 16 at device 26.0 on pci0 uhci0: LegSup =3D 0x2f00 usbus0 on uhci0 uhci1: <Intel 82801JI (ICH10) USB controller USB-E> port 0xb480-0xb49f irq = 21 at device 26.1 on pci0 uhci1: LegSup =3D 0x2f00 usbus1 on uhci1 uhci2: <Intel 82801JI (ICH10) USB controller USB-F> port 0xb400-0xb41f irq = 19 at device 26.2 on pci0 uhci2: LegSup =3D 0x2f00 usbus2 on uhci2 ehci0: <Intel 82801JI (ICH10) USB 2.0 controller USB-B> mem 0xfbdf4000-0xfb= df43ff irq 18 at device 26.7 on pci0 usbus3: EHCI version 1.0 usbus3 on ehci0 uhci3: <Intel 82801JI (ICH10) USB controller USB-A> port 0xc000-0xc01f irq = 23 at device 29.0 on pci0 uhci3: LegSup =3D 0x2f00 usbus4 on uhci3 uhci4: <Intel 82801JI (ICH10) USB controller USB-B> port 0xbc00-0xbc1f irq = 19 at device 29.1 on pci0 uhci4: LegSup =3D 0x2f00 usbus5 on uhci4 uhci5: <Intel 82801JI (ICH10) USB controller USB-C> port 0xb880-0xb89f irq = 18 at device 29.2 on pci0 uhci5: LegSup =3D 0x2f00 usbus6 on uhci5 ehci1: <Intel 82801JI (ICH10) USB 2.0 controller USB-A> mem 0xfbdf6000-0xfb= df63ff irq 23 at device 29.7 on pci0 usbus7: EHCI version 1.0 usbus7 on ehci1 pcib4: <ACPI PCI-PCI bridge> at device 30.0 on pci0 pci1: <ACPI PCI bus> on pcib4 vgapci0: <VGA-compatible display> port 0xdc00-0xdc7f mem 0xfb000000-0xfb7ff= fff,0xfafe0000-0xfaffffff irq 16 at device 5.0 on pci1 vgapci0: Boot video device isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 ahci0: <Intel ICH10 AHCI SATA controller> port 0xcc00-0xcc07,0xc880-0xc883,= 0xc800-0xc807,0xc480-0xc483,0xc400-0xc41f mem 0xfbdfa000-0xfbdfa7ff irq 19 = at device 31.2 on pci0 ahci0: AHCI v1.20 with 6 3Gbps ports, Port Multiplier not supported ahcich0: <AHCI channel> at channel 0 on ahci0 ahcich1: <AHCI channel> at channel 1 on ahci0 ahcich2: <AHCI channel> at channel 2 on ahci0 ahcich3: <AHCI channel> at channel 3 on ahci0 ahcich4: <AHCI channel> at channel 4 on ahci0 ahcich5: <AHCI channel> at channel 5 on ahci0 ahciem0: <AHCI enclosure management bridge> on ahci0 acpi_button0: <Power Button> on acpi0 ipmi0: <IPMI System Interface> port 0xca2,0xca6 on acpi0 ipmi0: KCS mode found at io 0xca2 on acpi qpi0: <QPI system bus> on motherboard pcib5: <QPI Host-PCI bridge> pcibus 255 on qpi0 pci255: <PCI bus> on pcib5 pcib6: <QPI Host-PCI bridge> pcibus 254 on qpi0 pci254: <PCI bus> on pcib6 orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0x= c9fff,0xca000-0xcafff on isa0 sc0: <System console> at flags 0x100 on isa0 sc0: CGA <16 virtual consoles, flags=3D0x300> vga0: <Generic ISA VGA> at port 0x3d0-0x3db iomem 0xb8000-0xbffff on isa0 atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] uart0: <Non-standard ns8250 class UART with FIFOs> at port 0x3f8-0x3ff irq = 4 on isa0 uart0: console (115200,n,8,1) coretemp0: <CPU On-Die Thermal Sensors> on cpu0 est0: <Enhanced SpeedStep Frequency Control> on cpu0 p4tcc0: <CPU Frequency Thermal Control> on cpu0 coretemp1: <CPU On-Die Thermal Sensors> on cpu1 est1: <Enhanced SpeedStep Frequency Control> on cpu1 p4tcc1: <CPU Frequency Thermal Control> on cpu1 coretemp2: <CPU On-Die Thermal Sensors> on cpu2 est2: <Enhanced SpeedStep Frequency Control> on cpu2 p4tcc2: <CPU Frequency Thermal Control> on cpu2 coretemp3: <CPU On-Die Thermal Sensors> on cpu3 est3: <Enhanced SpeedStep Frequency Control> on cpu3 p4tcc3: <CPU Frequency Thermal Control> on cpu3 coretemp4: <CPU On-Die Thermal Sensors> on cpu4 est4: <Enhanced SpeedStep Frequency Control> on cpu4 p4tcc4: <CPU Frequency Thermal Control> on cpu4 coretemp5: <CPU On-Die Thermal Sensors> on cpu5 est5: <Enhanced SpeedStep Frequency Control> on cpu5 p4tcc5: <CPU Frequency Thermal Control> on cpu5 coretemp6: <CPU On-Die Thermal Sensors> on cpu6 est6: <Enhanced SpeedStep Frequency Control> on cpu6 p4tcc6: <CPU Frequency Thermal Control> on cpu6 coretemp7: <CPU On-Die Thermal Sensors> on cpu7 est7: <Enhanced SpeedStep Frequency Control> on cpu7 p4tcc7: <CPU Frequency Thermal Control> on cpu7 coretemp8: <CPU On-Die Thermal Sensors> on cpu8 est8: <Enhanced SpeedStep Frequency Control> on cpu8 p4tcc8: <CPU Frequency Thermal Control> on cpu8 coretemp9: <CPU On-Die Thermal Sensors> on cpu9 est9: <Enhanced SpeedStep Frequency Control> on cpu9 p4tcc9: <CPU Frequency Thermal Control> on cpu9 coretemp10: <CPU On-Die Thermal Sensors> on cpu10 est10: <Enhanced SpeedStep Frequency Control> on cpu10 p4tcc10: <CPU Frequency Thermal Control> on cpu10 coretemp11: <CPU On-Die Thermal Sensors> on cpu11 est11: <Enhanced SpeedStep Frequency Control> on cpu11 p4tcc11: <CPU Frequency Thermal Control> on cpu11 coretemp12: <CPU On-Die Thermal Sensors> on cpu12 est12: <Enhanced SpeedStep Frequency Control> on cpu12 p4tcc12: <CPU Frequency Thermal Control> on cpu12 coretemp13: <CPU On-Die Thermal Sensors> on cpu13 est13: <Enhanced SpeedStep Frequency Control> on cpu13 p4tcc13: <CPU Frequency Thermal Control> on cpu13 coretemp14: <CPU On-Die Thermal Sensors> on cpu14 est14: <Enhanced SpeedStep Frequency Control> on cpu14 p4tcc14: <CPU Frequency Thermal Control> on cpu14 coretemp15: <CPU On-Die Thermal Sensors> on cpu15 est15: <Enhanced SpeedStep Frequency Control> on cpu15 p4tcc15: <CPU Frequency Thermal Control> on cpu15 random: unblocking device. usbus0: 12Mbps Full Speed USB v1.0 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec usbus1: 12Mbps Full Speed USB v1.0 ugen0.1: <Intel> at usbus0 uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0 ugen1.1: <Intel> at usbus1 uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 ugen2.1: <Intel> at usbus2 uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2 ugen3.1: <Intel> at usbus3 uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3 usbus4: 12Mbps Full Speed USB v1.0 usbus5: 12Mbps Full Speed USB v1.0 ugen4.1: <Intel> at usbus4 uhub4: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4 ugen5.1: <Intel> at usbus5 uhub5: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus5 usbus6: 12Mbps Full Speed USB v1.0 usbus7: 480Mbps High Speed USB v2.0 ugen6.1: <Intel> at usbus6 uhub6: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus6 ugen7.1: <Intel> at usbus7 uhub7: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus7 ipmi0: IPMI device rev. 1, firmware rev. 3.00, version 2.0 uhub0: 2 ports with 2 removable, self powered ipmi0: Number of channels 5 ipmi0: Attached watchdog ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 uhub1: 2 ports with 2 removable, self powered ada0: <HITACHI HUA7250SBSUN500G 0944JWRUMF GK6OAC5A> ATA-7uhub2: 2 ports wi= th 2 removable, self powered SATA 2.x device ada0: Serial Number GTA460P6GWRUMF ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 476940MB (976773168 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad4 ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 uhub4: 2 ports with 2 removable, self powered ada1: <HITACHI HUA7250SBSUN500G 0911JS4H0F GK6OAC5A> ATA-7uhub5: 2 ports wi= th 2 removable, self powered SATA 2.x device ada1: Serial Number GTF402P6GS4H0F ada1: 300.000MB/s transfersuhub6: 2 ports with 2 removable, self powered (SATA 2.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 476940MB (976773168 512 byte sectors: 16H 63S/T 16383C) ada1: Previously was known as ad6 ada2 at ahcich2 bus 0 scbus2 target 0 lun 0 ada2: <Samsung SSD 850 PRO 512GB EXM01B6Q> ATA-9 SATA 3.x device ada2: Serial Number S1SXNSAFA06834H ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 512bytes) ada2: Command Queueing enabled ada2: 488386MB (1000215216 512 byte sectors: 16H 63S/T 16383C) ada2: Previously was known as ad8 ada3 at ahcich3 bus 0 scbus3 target 0 lun 0 ada3: <Samsung SSD 850 PRO 512GB EXM01B6Q> ATA-9 SATA 3.x device ada3: Serial Number S1SXNSAFA06835A ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 512bytes) ada3: Command Queueing enabled ada3: 488386MB (1000215216 512 byte sectors: 16H 63S/T 16383C) ada3: Previously was known as ad10 ses0 at ahciem0 bus 0 scbus6 target 0 lun 0 ses0: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device ses0: SEMB SES Device GEOM_MIRROR: Device mirror/swap launched (2/2). SMP: AP CPU #1 Launched! SMP: AP CPU #8 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #9 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #14 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #10 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #13 Launched! SMP: AP CPU #12 Launched! SMP: AP CPU #15 Launched! SMP: AP CPU #11 Launched! SMP: AP CPU #4 Launched! Timecounter "TSC-low" frequency 1133397512 Hz quality 1000 Root mount waiting for: usbus7 usbus3 Root mount waiting for: usbus7 usbus3 uhub3: 6 ports with 6 removable, self powered uhub7: 6 ports with 6 removable, self powered Root mount waiting for: usbus7 usbus3 ugen3.2: <American Megatrends Inc.> at usbus3 usb_alloc_device: set address 2 failed (USB_ERR_IOERROR, ignored) Root mount waiting for: usbus7 ugen7.2: <JetFlash> at usbus7 umass0: <JetFlash Mass Storage Device, class 0/0, rev 2.00/11.00, addr 2> o= n usbus7 umass0: SCSI over Bulk-Only; quirks =3D 0x0100 umass0:7:0:-1: Attached to scbus7 da0 at umass-sim0 bus 0 scbus7 target 0 lun 0 da0: <JetFlash Transcend 4GB 1100> Removable Direct Access SCSI-4 device=20 da0: Serial Number 82L68FGM1BTVJHR0 da0: 40.000MB/s transfers da0: 3861MB (7907328 512 byte sectors: 255H 63S/T 492C) da0: quirks=3D0x12<NO_6_BYTE,NO_RC16> ugen2.2: <American Megatrends Inc.> at usbus2 ukbd0: <Keyboard Interface> on usbus2 Trying to mount root from zfs:rpool/ROOT/r273903-10.1 []... kbd2 at ukbd0 ums0: <Mouse Interface> on usbus2 ums0: 3 buttons and [XY] coordinates ID=3D0 ukbd1: <vendor 0x04d9 product 0x1400, class 0/0, rev 1.10/1.43, addr 2> on = usbus4 kbd3 at ukbd1 ums1: <vendor 0x04d9 product 0x1400, class 0/0, rev 1.10/1.43, addr 2> on u= sbus4 ums1: 5 buttons and [XYZ] coordinates ID=3D1 ugen4.2: <vendor 0x04d9> at usbus4 (disconnected) ukbd1: at uhub4, port 2, addr 2 (disconnected) ums1: at uhub4, port 2, addr 2 (disconnected) ugen4.2: <vendor 0x04d9> at usbus4 ukbd1: <vendor 0x04d9 product 0x1400, class 0/0, rev 1.10/1.43, addr 2> on = usbus4 kbd3 at ukbd1 ums1: <vendor 0x04d9 product 0x1400, class 0/0, rev 1.10/1.43, addr 2> on u= sbus4 ums1: 5 buttons and [XYZ] coordinates ID=3D1 --=20 PGP-KeyID =3D 0xE401B671927D4A5C --Sig_/H3gB3vpPwbhWohn56fzs7vY Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCgAGBQJUWsB7AAoJEHBlTXxPsfWIkdwQAKFeuy3paHObKkPFvtFy47+2 Hk4two2A9PB1Wl838GDw0z0td9hWBcrrtUtXccHKqWmou2CtHwJ+YOy5KsOAw6Nh s+LKXPHn5GopZ1vg0mh9n4u1oS7BVjI46lsbGq9bNvgOkDv/B0KHBDT1rRQ9TOWh 6EviAINN5Gf343O8HF9LLW5Ho5hs3+oH5CEW6CRrtUR9afQ7t7htKxEstQUYkBzG HDQ55iArv/cZBWsM5M99WkjTfZ+4eZ5foYw3r0rJxlrGzQr/L3KNYHhbQz5dl+FN 3/gZb94gjIlP/w2cQBEKO2+6CO/b9fRnk51yu/tLiK/ZHYNmrWHrpBqb2OFqlqfI 9x04pKU4xvI4alW/Rr7el/kyFHyE8EuqcPafbGMmwrEBSV3BfCHPIcDJ+VXw/wEI dwHmp6Ymp3Kyr5jBEtWo1YoiheYDx/Bqq8frFLCSuvc2mddN+p5ORAJVc5M0IYkR IEXw77KNQQS8w8bTbeIqO6lPGm8CKjSUqDqOK3PvC43Gf5QSYKazZXR2VpTj7P5l jFHmjhjLuWY3XeGhI2QyC4arEzVAZ/GaPqRqDISnwMLiChda3drbZFSE4OmdqfWf //kty7E1hZkvmSLWcRZXNh7V8Z35Op9d93vwcIdJx4XqQIxCqk3cv/A/A6s029hA TSqcQkUCz6c3+b0hR53r =eTfE -----END PGP SIGNATURE----- --Sig_/H3gB3vpPwbhWohn56fzs7vY--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20141106012739.509b96b5>