Date: Mon, 13 Jun 2005 12:02:14 +0200 From: Geoffroy Desvernay <dgeo@egim-mrs.fr> To: freebsd-stable@freebsd.org Subject: kernel bug (ufs2?) on a dell 2600 Message-ID: <42AD59A6.3080309@egim-mrs.fr>
next in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format. --------------ms000906060407050808080506 Content-Type: multipart/mixed; boundary="------------080202090501040804060303" This is a multi-part message in MIME format. --------------080202090501040804060303 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit This server (FreeBSD 5.4 RELENG) is crashing once a week or more since 5.4 (maybe before). It may be related with a full filesystem: I'm using snapshots on this server (using http://people.freebsd.org/~rse/snapshot/), and crack has occured after (~30mins) a snapshot that fills up to 100% the filesystem. Attached the dmesg and kgdb logs. I'm not so hacker, but hope that it can help to resolve this bug. --------------080202090501040804060303 Content-Type: text/plain; name="LOG.kgdb" Content-Transfer-Encoding: 8bit Content-Disposition: inline; filename="LOG.kgdb" [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". #0 doadump () at pcpu.h:160 160 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); (kgdb) bt full #0 doadump () at pcpu.h:160 No locals. #1 0xc06878d6 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:410 first_buf_printf = 1 #2 0xc0687cc4 in panic (fmt=0xc091a1ae "initiate_write_inodeblock_ufs2: already started") at /usr/src/sys/kern/kern_shutdown.c:566 td = (struct thread *) 0xc3641c00 bootopt = 260 newpanic = 0 ap = 0xc3641c00 "\\\214\220Ã -XÃ" buf = "initiate_write_inodeblock_ufs2: already started", '\0' <repeats 208 times> #3 0xc080ef5f in initiate_write_inodeblock_ufs2 (inodedep=0xc5be0280, bp=0x0) at /usr/src/sys/ufs/ffs/ffs_softdep.c:3781 adp = (struct allocdirect *) 0xd75bd13c lastadp = (struct allocdirect *) 0x1000 dp = (struct ufs2_dinode *) 0x0 fs = (struct fs *) 0xc21f6730 i = Unhandled dwarf expression opcode 0x93 (kgdb) quit --------------080202090501040804060303 Content-Type: text/plain; name="dmesg.boot" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="dmesg.boot" Copyright (c) 1992-2005 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.4-STABLE #0: Mon Jun 6 18:51:49 CEST 2005 root@zlip.egim-mrs.fr:/usr/obj/usr/src/sys/ZLIP Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(TM) CPU 2.40GHz (2392.29-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf29 Stepping = 9 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Hyperthreading: 2 logical CPUs real memory = 2147287040 (2047 MB) avail memory = 2095828992 (1998 MB) ACPI APIC Table: <DELL PE2600 > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 6 cpu3 (AP): APIC ID: 7 ioapic0: Changing APIC ID to 8 ioapic1: Changing APIC ID to 9 ioapic2: Changing APIC ID to 10 ioapic2: WARNING: intbase 72 != expected base 48 ioapic3: Changing APIC ID to 11 ioapic3: WARNING: intbase 120 != expected base 96 ioapic4: Changing APIC ID to 12 ioapic0 <Version 2.0> irqs 0-23 on motherboard ioapic1 <Version 2.0> irqs 24-47 on motherboard ioapic2 <Version 2.0> irqs 72-95 on motherboard ioapic3 <Version 2.0> irqs 120-143 on motherboard ioapic4 <Version 2.0> irqs 144-167 on motherboard npx0: <math processor> on motherboard npx0: INT 16 interface acpi0: <DELL PE2600> on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-safe" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 cpu0: <ACPI CPU> on acpi0 cpu1: <ACPI CPU> on acpi0 cpu2: <ACPI CPU> on acpi0 cpu3: <ACPI CPU> on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0 pci1: <ACPI PCI bus> on pcib1 pci1: <base peripheral, interrupt controller> at device 28.0 (no driver attached) pcib2: <ACPI PCI-PCI bridge> at device 29.0 on pci1 pci2: <ACPI PCI bus> on pcib2 em0: <Intel(R) PRO/1000 Network Connection, Version - 1.7.35> port 0xece0-0xecff mem 0xfdec0000-0xfdedffff,0xfdee0000-0xfdefffff irq 24 at device 2.0 on pci2 em0: Ethernet address: 00:02:b3:d4:d3:a2 em0: Speed:N/A Duplex:N/A pci1: <base peripheral, interrupt controller> at device 30.0 (no driver attached) pcib3: <ACPI PCI-PCI bridge> at device 31.0 on pci1 pci3: <ACPI PCI bus> on pcib3 em1: <Intel(R) PRO/1000 Network Connection, Version - 1.7.35> port 0xdce0-0xdcff mem 0xfdcc0000-0xfdcdffff,0xfdce0000-0xfdcfffff irq 28 at device 1.0 on pci3 em1: Ethernet address: 00:0b:db:92:0a:e4 em1: Speed:N/A Duplex:N/A pcib4: <ACPI PCI-PCI bridge> at device 3.0 on pci0 pci4: <ACPI PCI bus> on pcib4 pci4: <base peripheral, interrupt controller> at device 28.0 (no driver attached) pcib5: <ACPI PCI-PCI bridge> at device 29.0 on pci4 pci5: <ACPI PCI bus> on pcib5 pci4: <base peripheral, interrupt controller> at device 30.0 (no driver attached) pcib6: <ACPI PCI-PCI bridge> at device 31.0 on pci4 pci6: <ACPI PCI bus> on pcib6 bge0: <Broadcom BCM5703 Gigabit Ethernet, ASIC rev. 0x1002> mem 0xfd8f0000-0xfd8fffff irq 80 at device 3.0 on pci6 miibus0: <MII bus> on bge0 brgphy0: <BCM5703 10/100/1000baseTX PHY> on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto bge0: Ethernet address: 00:10:18:06:39:68 pcib7: <ACPI PCI-PCI bridge> at device 4.0 on pci0 pci7: <ACPI PCI bus> on pcib7 pci7: <base peripheral, interrupt controller> at device 28.0 (no driver attached) pcib8: <ACPI PCI-PCI bridge> at device 29.0 on pci7 pci8: <ACPI PCI bus> on pcib8 amr0: <LSILogic MegaRAID 1.51> mem 0xfebf0000-0xfebfffff irq 120 at device 8.0 on pci8 amr0: <LSILogic PERC 4/Di> Firmware 250O, BIOS 1.06, 128MB RAM pci7: <base peripheral, interrupt controller> at device 30.0 (no driver attached) pcib9: <ACPI PCI-PCI bridge> at device 31.0 on pci7 pci10: <ACPI PCI bus> on pcib9 ahc0: <Adaptec 3960D Ultra160 SCSI adapter> port 0xbc00-0xbcff mem 0xfd3ff000-0xfd3fffff irq 148 at device 7.0 on pci10 aic7899: Ultra160 Wide Channel A, SCSI Id=7, 32/253 SCBs ahc1: <Adaptec 3960D Ultra160 SCSI adapter> port 0xb800-0xb8ff mem 0xfd3fe000-0xfd3fefff irq 149 at device 7.1 on pci10 aic7899: Ultra160 Wide Channel B, SCSI Id=7, 32/253 SCBs uhci0: <Intel 82801CA/CAM (ICH3) USB controller USB-A> port 0x9ce0-0x9cff irq 16 at device 29.0 on pci0 usb0: <Intel 82801CA/CAM (ICH3) USB controller USB-A> on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered pcib10: <ACPI PCI-PCI bridge> at device 30.0 on pci0 pci11: <ACPI PCI bus> on pcib10 pci11: <display, VGA> at device 4.0 (no driver attached) isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <Intel ICH3 UDMA100 controller> port 0xfc00-0xfc0f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 31.1 on pci0 ata0: channel #0 on atapci0 ata1: channel #1 on atapci0 fdc0: <floppy drive controller> port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 psm0: <PS/2 Mouse> irq 12 on atkbdc0 psm0: model IntelliMouse Explorer, device ID 4 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A ppc0: <ECP parallel printer port> port 0x778-0x77f,0x378-0x37f irq 7 drq 1 on acpi0 ppc0: Generic chipset (ECP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/8 bytes threshold ppbus0: <Parallel port bus> on ppc0 plip0: <PLIP network interface> on ppbus0 lpt0: <Printer> on ppbus0 lpt0: Interrupt-driven port ppi0: <Parallel I/O> on ppbus0 orm0: <ISA Option ROMs> at iomem 0xec000-0xeffff,0xc0000-0xc7fff on isa0 pmtimer0 on isa0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounters tick every 10.000 msec acd0: CDROM <SAMSUNG CD-ROM SN-124/N103> at ata0-master PIO4 Waiting 15 seconds for SCSI devices to settle amrd0: <LSILogic MegaRAID logical drive> on amr0 amrd0: 69758MB (142864384 sectors) RAID 1 (optimal) ses0 at amr0 bus 0 target 6 lun 0 ses0: <PE/PV 1x6 SCSI BP 1.1> Fixed Processor SCSI-2 device ses0: SAF-TE Compliant Device da0 at ahc0 bus 0 target 0 lun 0 da0: <transtec 0001> Fixed Direct Access SCSI-3 device da0: 160.000MB/s transfers (80.000MHz, offset 31, 16bit), Tagged Queueing Enabled da0: 200000MB (409600000 512 byte sectors: 255H 63S/T 25496C) da1 at ahc0 bus 0 target 1 lun 0 da1: <transtec 0001> Fixed Direct Access SCSI-3 device da1: 160.000MB/s transfers (80.000MHz, offset 31, 16bit), Tagged Queueing Enabled da1: 200000MB (409600000 512 byte sectors: 255H 63S/T 25496C) da2 at ahc0 bus 0 target 2 lun 0 da2: <transtec 0001> Fixed Direct Access SCSI-3 device da2: 160.000MB/s transfers (80.000MHz, offset 31, 16bit), Tagged Queueing Enabled da2: 200000MB (409600000 512 byte sectors: 255H 63S/T 25496C) da3 at ahc0 bus 0 target 3 lun 0 da3: <transtec 0001> Fixed Direct Access SCSI-3 device da3: 160.000MB/s transfers (80.000MHz, offset 31, 16bit), Tagged Queueing Enabled da3: 300000MB (614400000 512 byte sectors: 255H 63S/T 38244C) da4 at ahc0 bus 0 target 4 lun 0 da4: <transtec 0001> Fixed Direct Access SCSI-3 device da4: 160.000MB/s transfers (80.000MHz, offset 31, 16bit), Tagged Queueing Enabled da4: 266856MB (546521088 512 byte sectors: 255H 63S/T 34019C) SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! Mounting root from ufs:/dev/amrd0s1a WARNING: / was not properly dismounted WARNING: /tmp was not properly dismounted WARNING: /usr was not properly dismounted WARNING: /var was not properly dismounted /var: mount pending error: blocks 156 files 1 WARNING: /var/crash was not properly dismounted WARNING: /anonftp was not properly dismounted WARNING: /jukebox was not properly dismounted WARNING: /anonftp/pub2 was not properly dismounted WARNING: /savesm2 was not properly dismounted /savesm2: mount pending error: blocks 3230304 files 1 WARNING: /sav was not properly dismounted em1: Link is up 1000 Mbps Full Duplex --------------080202090501040804060303-- --------------ms000906060407050808080506 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIOazCC A9swggLDoAMCAQICAQIwDQYJKoZIhvcNAQEEBQAwTzELMAkGA1UEBhMCRlIxDDAKBgNVBAoT A0NSVTESMBAGA1UEAxMJYWMtcmFjaW5lMR4wHAYJKoZIhvcNAQkBFg9jYS1hZG1pbkBjcnUu ZnIwHhcNMDMwNTE2MDkzMTE3WhcNMTIxMjE0MDkzMTE3WjBUMQswCQYDVQQGEwJGUjEMMAoG A1UEChMDQ1JVMRcwFQYDVQQDEw5hYy11dGlsaXNhdGV1cjEeMBwGCSqGSIb3DQEJARYPY2Et YWRtaW5AY3J1LmZyMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtXGYMirQ2feu 6b4FDrH8PRwIyjfXPSXaG33GoatYJIbOmxAwnGbwVBng3WDzoW5X45RWq93nOxSdx4MTsGQJ pBihob2h4XeGpL54WlLd+kvOxvp/0OzLYYcESI3dpVRjf6jB8Ggc7BUlE+ptIjyXAVTNKz1o 6aiLSR6G1zG5cidq3BAXjlpfvbX9EWfICcnmLMSQ29oam857EsAJRuGHNajdPEMUlsioEIf2 Zf4YL9pa5rJ5Efs7NTxu8ffICGYlCynJXJTC461sRrub8s6SOM69NTw8EM+1A3ewd5WyGkZe QSG7SJyJY8Lg8u4CMoflOjP3LSYhCLI1Ic0gX08dkQIDAQABo4G8MIG5MA8GA1UdEwEB/wQF MAMBAf8wHQYDVR0OBBYEFBUKfCi1rKa81/0yMx15vbNJCjGQMHcGA1UdIwRwMG6AFK8vz0qV BLbeJLMJqVU+XA4c+HEuoVOkUTBPMQswCQYDVQQGEwJGUjEMMAoGA1UEChMDQ1JVMRIwEAYD VQQDEwlhYy1yYWNpbmUxHjAcBgkqhkiG9w0BCQEWD2NhLWFkbWluQGNydS5mcoIBADAOBgNV HQ8BAf8EBAMCAQYwDQYJKoZIhvcNAQEEBQADggEBAMG/+LofN2vISFWieXrHsFsIrM/2WUhf zAcE2/Ga+/UWvJaDJO9RlbUNN3l+f40L1yUJW9kZqY4XMBcbJ6jbpmeJ+zTf+zveAD/sl9ZN qoC931HFLq/TGjZSIQxX4Nxad9DFbok66X1qfRZF4+beaz2zjrF07LvjWX6201Rc6zQhNyTy UdO/wYY0EBYu0CFrTK4RxS6d2LsyO1p1Xj7oMCa9Utmu/igZoXJYL709POBKHJoYNU+LI6iJ 8aYcZupJ/GXVxfj68L1KP2i8cM3a570bJCDbsKcchzZqsvII5cewU7KKPH4fyMyfsA39tJ0V BMQyvdWZHpS75DN0eB5UVzUwggVCMIIEKqADAgECAgIEkzANBgkqhkiG9w0BAQQFADBUMQsw CQYDVQQGEwJGUjEMMAoGA1UEChMDQ1JVMRcwFQYDVQQDEw5hYy11dGlsaXNhdGV1cjEeMBwG CSqGSIb3DQEJARYPY2EtYWRtaW5AY3J1LmZyMB4XDTA1MDMyMzEwMjEwMloXDTA2MDMyMzEw MjEwMlowXjELMAkGA1UEBhMCRlIxETAPBgNVBAoTCDAxMzM3NzRHMRswGQYDVQQDExJHZW9m ZnJveSBEZXN2ZXJuYXkxHzAdBgkqhkiG9w0BCQEWEGRnZW9AZWdpbS1tcnMuZnIwggEiMA0G CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCms7qWSkTwUJvQq2Mu2ghrvv9AXcaDASOKblYa lTFCmi5fYR9FbTkK+/F7+q7DsEc+cFk12S7mTk/n0M6zyqqTYZ8P1CpAqOj9L8vYHHoRTmi+ nLr5SkZ74KdERxLgFsh5A7xz4+aACOVwXweTWeaGZ7qkwZCduRpY+t/l88AQXT6PA2Ao2cEc 1jsAFqnCGAyo6M1oVB+UikZjBi8DH+ZJFCDt53OFSrBGCxwb0O1QSmUeIBCQpK32N7vy2+Wo uWNrN232Qv9slhb9p0pP8ki1kXdmgQqXAzmsT8sa4jmyTWVyhSvVXdjRff6LMtxrD+58H4aN A3VBh5zG0TwMw/yvAgMBAAGjggISMIICDjAMBgNVHRMBAf8EAjAAMBEGCWCGSAGG+EIBAQQE AwIEsDAOBgNVHQ8BAf8EBAMCBeAwEwYDVR0lBAwwCgYIKwYBBQUHAwIwGwYDVR0RBBQwEoEQ ZGdlb0BlZ2ltLW1ycy5mcjBwBglghkgBhvhCAQ0EYxZhQ2VydGlmaWNhdCBhYy11dGlsaXNh dGV1ci4gUG91ciB0b3V0ZSBpbmZvcm1hdGlvbiBzZSByZXBvcnRlciDgIGh0dHA6Ly9pZ2Mu Y3J1LmZyL2FjLXV0aWxpc2F0ZXVyLzAdBgNVHQ4EFgQU2L2xnBQv4ehMXBUG53tonqPd6Gcw dwYDVR0jBHAwboAUFQp8KLWsprzX/TIzHXm9s0kKMZChU6RRME8xCzAJBgNVBAYTAkZSMQww CgYDVQQKEwNDUlUxEjAQBgNVBAMTCWFjLXJhY2luZTEeMBwGCSqGSIb3DQEJARYPY2EtYWRt aW5AY3J1LmZyggECME8GA1UdHwRIMEYwRKBCoECGPmh0dHA6Ly9pZ2MuY3J1LmZyL2NnaS1i aW4vbG9hZGNybD9DQT1hYy11dGlsaXNhdGV1ciZmb3JtYXQ9REVSME4GA1UdIARHMEUwQwYO KQMGAQQAAbdfAwECAAQwMTAvBggrBgEFBQcCARYjaHR0cDovL2lnYy5jcnUuZnIvcmVmZXJl bmNlcy9wYy5wZGYwDQYJKoZIhvcNAQEEBQADggEBAEiz6Id63DiwxH8SKnZKOfJoAlLmDJtu L6NS8r6KjSANcueo5wNVw2MYFs4v0ehwabtYbdjtjnPN325jqf246wZ+3c5LfBl7hQEkfTpy QGn6uOFlrCYnephGfxES4qJfFJDZx5Ugu1wYGlQ3Ojr6VNRrl8esWSHfYI5VrrX/Z1f835or LaYJ2Osdn9NNlV8wMvoMpZQwZEOjHI3fiTjneVwPQY3yoxktscU8NWEhD/HtKnAKqvp9/VjU 2Sci8fMj0DeJErsIVmbIYjX6adRi+OxnsiqB9wW2zHC1lgbBVIy1ypJHtOnbrxGzcb/4D3mJ 5+6deMUdyn6a8xiP64Tb6tEwggVCMIIEKqADAgECAgIEkzANBgkqhkiG9w0BAQQFADBUMQsw CQYDVQQGEwJGUjEMMAoGA1UEChMDQ1JVMRcwFQYDVQQDEw5hYy11dGlsaXNhdGV1cjEeMBwG CSqGSIb3DQEJARYPY2EtYWRtaW5AY3J1LmZyMB4XDTA1MDMyMzEwMjEwMloXDTA2MDMyMzEw MjEwMlowXjELMAkGA1UEBhMCRlIxETAPBgNVBAoTCDAxMzM3NzRHMRswGQYDVQQDExJHZW9m ZnJveSBEZXN2ZXJuYXkxHzAdBgkqhkiG9w0BCQEWEGRnZW9AZWdpbS1tcnMuZnIwggEiMA0G CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCms7qWSkTwUJvQq2Mu2ghrvv9AXcaDASOKblYa lTFCmi5fYR9FbTkK+/F7+q7DsEc+cFk12S7mTk/n0M6zyqqTYZ8P1CpAqOj9L8vYHHoRTmi+ nLr5SkZ74KdERxLgFsh5A7xz4+aACOVwXweTWeaGZ7qkwZCduRpY+t/l88AQXT6PA2Ao2cEc 1jsAFqnCGAyo6M1oVB+UikZjBi8DH+ZJFCDt53OFSrBGCxwb0O1QSmUeIBCQpK32N7vy2+Wo uWNrN232Qv9slhb9p0pP8ki1kXdmgQqXAzmsT8sa4jmyTWVyhSvVXdjRff6LMtxrD+58H4aN A3VBh5zG0TwMw/yvAgMBAAGjggISMIICDjAMBgNVHRMBAf8EAjAAMBEGCWCGSAGG+EIBAQQE AwIEsDAOBgNVHQ8BAf8EBAMCBeAwEwYDVR0lBAwwCgYIKwYBBQUHAwIwGwYDVR0RBBQwEoEQ ZGdlb0BlZ2ltLW1ycy5mcjBwBglghkgBhvhCAQ0EYxZhQ2VydGlmaWNhdCBhYy11dGlsaXNh dGV1ci4gUG91ciB0b3V0ZSBpbmZvcm1hdGlvbiBzZSByZXBvcnRlciDgIGh0dHA6Ly9pZ2Mu Y3J1LmZyL2FjLXV0aWxpc2F0ZXVyLzAdBgNVHQ4EFgQU2L2xnBQv4ehMXBUG53tonqPd6Gcw dwYDVR0jBHAwboAUFQp8KLWsprzX/TIzHXm9s0kKMZChU6RRME8xCzAJBgNVBAYTAkZSMQww CgYDVQQKEwNDUlUxEjAQBgNVBAMTCWFjLXJhY2luZTEeMBwGCSqGSIb3DQEJARYPY2EtYWRt aW5AY3J1LmZyggECME8GA1UdHwRIMEYwRKBCoECGPmh0dHA6Ly9pZ2MuY3J1LmZyL2NnaS1i aW4vbG9hZGNybD9DQT1hYy11dGlsaXNhdGV1ciZmb3JtYXQ9REVSME4GA1UdIARHMEUwQwYO KQMGAQQAAbdfAwECAAQwMTAvBggrBgEFBQcCARYjaHR0cDovL2lnYy5jcnUuZnIvcmVmZXJl bmNlcy9wYy5wZGYwDQYJKoZIhvcNAQEEBQADggEBAEiz6Id63DiwxH8SKnZKOfJoAlLmDJtu L6NS8r6KjSANcueo5wNVw2MYFs4v0ehwabtYbdjtjnPN325jqf246wZ+3c5LfBl7hQEkfTpy QGn6uOFlrCYnephGfxES4qJfFJDZx5Ugu1wYGlQ3Ojr6VNRrl8esWSHfYI5VrrX/Z1f835or LaYJ2Osdn9NNlV8wMvoMpZQwZEOjHI3fiTjneVwPQY3yoxktscU8NWEhD/HtKnAKqvp9/VjU 2Sci8fMj0DeJErsIVmbIYjX6adRi+OxnsiqB9wW2zHC1lgbBVIy1ypJHtOnbrxGzcb/4D3mJ 5+6deMUdyn6a8xiP64Tb6tExggMOMIIDCgIBATBaMFQxCzAJBgNVBAYTAkZSMQwwCgYDVQQK EwNDUlUxFzAVBgNVBAMTDmFjLXV0aWxpc2F0ZXVyMR4wHAYJKoZIhvcNAQkBFg9jYS1hZG1p bkBjcnUuZnICAgSTMAkGBSsOAwIaBQCgggGJMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEw HAYJKoZIhvcNAQkFMQ8XDTA1MDYxMzEwMDIxNFowIwYJKoZIhvcNAQkEMRYEFPgnMVOgHYKI Wd5Cb3iDMrQWQuvWMFIGCSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwIC AgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMGkGCSsGAQQBgjcQ BDFcMFowVDELMAkGA1UEBhMCRlIxDDAKBgNVBAoTA0NSVTEXMBUGA1UEAxMOYWMtdXRpbGlz YXRldXIxHjAcBgkqhkiG9w0BCQEWD2NhLWFkbWluQGNydS5mcgICBJMwawYLKoZIhvcNAQkQ AgsxXKBaMFQxCzAJBgNVBAYTAkZSMQwwCgYDVQQKEwNDUlUxFzAVBgNVBAMTDmFjLXV0aWxp c2F0ZXVyMR4wHAYJKoZIhvcNAQkBFg9jYS1hZG1pbkBjcnUuZnICAgSTMA0GCSqGSIb3DQEB AQUABIIBAERCg2GMwz4T2+9gwtwISBOFPd8yVaNcKFHobq6maiSpwLEWMCx3HFb2MpVMUz5M XsmANyc7T2tEXTUpzAQkvdSFtTzejp1P0/aKNwrVVoWTTKMYDPWXVdezaF6qxONc0EFTQo7A SIbNdf4SQtJaba0wxVTxJjMq1S44wmiQ3m0LdpTDYpEstocCiDrFvjsa3aOAafP+rKs6MbRd e5GgMDtiCv2Bx+vv8n6Z/QMoeHdVb5/MJpkPW38q48PSsug87hkMi5T2BwJHYheukxmvi/ad XODn7lUVzzmi1uCCWvy+X5gaLJmSx/H9YYykvQMhLVLF6nJwDqssBLNLYrX4clEAAAAAAAA= --------------ms000906060407050808080506--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?42AD59A6.3080309>