Date: Tue, 6 Feb 2007 02:28:00 +0300 From: Oleg Bulyzhin <oleg@freebsd.org> To: Robin Gruyters <r.gruyters@yirdis.nl> Cc: freebsd-net@freebsd.org Subject: Re: [Fwd: Re: bge Ierr rate increase from 5.3R -> 6.1R] Message-ID: <20070205232800.GA45487@lath.rinet.ru> In-Reply-To: <20070125170532.c9c2374hkwk4oc4k@server.yirdis.net> References: <20070125170532.c9c2374hkwk4oc4k@server.yirdis.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--k+w/mQv8wyuph6w0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
On Thu, Jan 25, 2007 at 05:05:32PM +0100, Robin Gruyters wrote:
> On 13-Jan-2007 Oleg Bulyzhin wrote:
> >
> >> Could you please test attached patch? It should fix ierrs issue
> >and should not
> >> break link events (would be fine to test it: unplug/plug cable,
> >try to change
> >> media with ifconfig, change media on other end of wire).
> >>
> Hi ya,
>
> Just wondering will this patch/update be available for RELENG_6? I
> have the same problems here with 3 of our servers.
>
> Here is dmesg from our development server:
>
> [...]
> Copyright (c) 1992-2007 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 6.2-RELEASE #1: Mon Jan 15 15:34:19 CET 2007
> root@server.yirdis.net:/data/obj/data/src_6_2/sys/YIRDIS
> Timecounter "i8254" frequency 1193182 Hz quality 0
> CPU: Intel(R) Xeon(TM) CPU 3.00GHz (3000.12-MHz 686-class CPU)
> Origin = "GenuineIntel" Id = 0xf41 Stepping = 1
>
> Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
> Features2=0x641d<SSE3,RSVD2,MON,DS_CPL,CNTX-ID,CX16,<b14>>
> AMD Features=0x20000000<LM>
> Logical CPUs per core: 2
> real memory = 2147430400 (2047 MB)
> avail memory = 2100547584 (2003 MB)
> ACPI APIC Table: <HP 00000083>
> Security auditing service present
> BSM auditing present
> ioapic1: Changing APIC ID to 9
> ioapic0 <Version 2.0> irqs 0-23 on motherboard
> ioapic1 <Version 2.0> irqs 24-47 on motherboard
> ioapic2 <Version 2.0> irqs 48-71 on motherboard
> ioapic3 <Version 2.0> irqs 72-95 on motherboard
> acpi0: <HP P52> on motherboard
> acpi0: Power Button (fixed)
> Timecounter "ACPI-safe" frequency 3579545 Hz quality 1000
> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x908-0x90b on acpi0
> cpu0: <ACPI CPU> on acpi0
> pcib0: <ACPI Host-PCI bridge> on acpi0
> pci0: <ACPI PCI bus> on pcib0
> pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
> pci13: <ACPI PCI bus> on pcib1
> pcib2: <ACPI PCI-PCI bridge> at device 4.0 on pci0
> pci6: <ACPI PCI bus> on pcib2
> pcib3: <ACPI PCI-PCI bridge> at device 0.0 on pci6
> pci7: <ACPI PCI bus> on pcib3
> pcib4: <ACPI PCI-PCI bridge> at device 0.2 on pci6
> pci10: <ACPI PCI bus> on pcib4
> pcib5: <PCI-PCI bridge> at device 1.0 on pci10
> pci11: <PCI bus> on pcib5
> ste0: <D-Link DL10050 10/100BaseTX> port 0x5000-0x507f irq 72 at
> device 4.0 on pci11
> miibus0: <MII bus> on ste0
> ukphy0: <Generic IEEE 802.3u media interface> on miibus0
> ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
> ste0: Ethernet address: 00:0d:88:fc:c8:cc
> ste1: <D-Link DL10050 10/100BaseTX> port 0x5080-0x50ff irq 73 at
> device 5.0 on pci11
> miibus1: <MII bus> on ste1
> ukphy1: <Generic IEEE 802.3u media interface> on miibus1
> ukphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
> ste1: Ethernet address: 00:0d:88:fc:c8:cd
> ste2: <D-Link DL10050 10/100BaseTX> port 0x5400-0x547f irq 74 at
> device 6.0 on pci11
> miibus2: <MII bus> on ste2
> ukphy2: <Generic IEEE 802.3u media interface> on miibus2
> ukphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
> ste2: Ethernet address: 00:0d:88:fc:c8:ce
> ste3: <D-Link DL10050 10/100BaseTX> port 0x5480-0x54ff irq 75 at
> device 7.0 on pci11
> miibus3: <MII bus> on ste3
> ukphy3: <Generic IEEE 802.3u media interface> on miibus3
> ukphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
> ste3: Ethernet address: 00:0d:88:fc:c8:cf
> pcib6: <ACPI PCI-PCI bridge> at device 6.0 on pci0
> pci3: <ACPI PCI bus> on pcib6
> pcib7: <ACPI PCI-PCI bridge> at device 28.0 on pci0
> pci2: <ACPI PCI bus> on pcib7
> ciss0: <HP Smart Array 6i> port 0x4000-0x40ff mem
> 0xfdff0000-0xfdff1fff,0xfdf80000-0xfdfbffff irq 24 at device 1.0 on pci2
> ciss0: [GIANT-LOCKED]
> bge0: <Broadcom BCM5704 B0, ASIC rev. 0x2100> mem
> 0xfdf70000-0xfdf7ffff irq 25 at device 2.0 on pci2
> miibus4: <MII bus> on bge0
> brgphy0: <BCM5704 10/100/1000baseTX PHY> on miibus4
> brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,
> 1000baseTX-FDX, auto
> bge0: Ethernet address: 00:12:79:94:ed:12
> bge1: <Broadcom BCM5704 B0, ASIC rev. 0x2100> mem
> 0xfdf60000-0xfdf6ffff irq 26 at device 2.1 on pci2
> miibus5: <MII bus> on bge1
> brgphy1: <BCM5704 10/100/1000baseTX PHY> on miibus5
> brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,
> 1000baseTX-FDX, auto
> bge1: Ethernet address: 00:12:79:94:ed:11
> pci0: <serial bus, USB> at device 29.0 (no driver attached)
> pci0: <serial bus, USB> at device 29.1 (no driver attached)
> pci0: <base peripheral> at device 29.4 (no driver attached)
> pci0: <base peripheral, interrupt controller> at device 29.5 (no
> driver attached)
> pci0: <serial bus, USB> at device 29.7 (no driver attached)
> pcib8: <ACPI PCI-PCI bridge> at device 30.0 on pci0
> pci1: <ACPI PCI bus> on pcib8
> pci1: <display, VGA> at device 3.0 (no driver attached)
> pci1: <base peripheral> at device 4.0 (no driver attached)
> pci1: <base peripheral> at device 4.2 (no driver attached)
> isab0: <PCI-ISA bridge> at device 31.0 on pci0
> isa0: <ISA bus> on isab0
> atapci0: <Intel 6300ESB UDMA100 controller> port
> 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x500-0x50f irq 18 at device 31.1
> on pci0
> ata0: <ATA channel 0> on atapci0
> ata1: <ATA channel 1> on atapci0
> acpi_tz0: <Thermal Zone> on acpi0
> atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
> atkbd0: <AT Keyboard> irq 1 on atkbdc0
> kbd0 at atkbd0
> atkbd0: [GIANT-LOCKED]
> psm0: <PS/2 Mouse> irq 12 on atkbdc0
> psm0: [GIANT-LOCKED]
> psm0: model IntelliMouse Explorer, device ID 4
> sio0: <Standard PC COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
> sio0: type 16550A
> fdc0: <floppy drive controller (FDE)> port 0x3f2-0x3f5 irq 6 drq 2 on acpi0
> fdc0: [FAST]
> fd0: <1440-KB 3.5" drive> on fdc0 drive 0
> pmtimer0 on isa0
> orm0: <ISA Option ROMs> at iomem
> 0xc0000-0xc7fff,0xc8000-0xcbfff,0xee000-0xeffff on isa0
> sc0: <System console> at flags 0x100 on isa0
> sc0: VGA <16 virtual consoles, flags=0x300>
> sio1 at port 0x2f8-0x2ff irq 3 on isa0
> sio1: type 16550A
> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
> Timecounter "TSC" frequency 3000124702 Hz quality 800
> Timecounters tick every 1.000 msec
> acd0: CDROM <COMPAQ CD-ROM SN-124/N104> at ata0-master PIO4
> da0 at ciss0 bus 0 target 0 lun 0
> da0: <COMPAQ RAID 1 VOLUME OK> Fixed Direct Access SCSI-0 device
> da0: 135.168MB/s transfers
> da0: 69459MB (142253280 512 byte sectors: 255H 32S/T 17433C)
> Trying to mount root from ufs:/dev/da0s1a
> Accounting enabled
> [...]
>
> And here the netstat interface statistics:
> [...]
> Name Mtu Network Address Ipkts Ierrs Opkts
> Oerrs Coll
> ste0* 1500 <Link#1> 00:0d:88:fc:c8:cc 0 0 0
> 0 0
> ste1* 1500 <Link#2> 00:0d:88:fc:c8:cd 0 0 0
> 0 0
> ste2* 1500 <Link#3> 00:0d:88:fc:c8:ce 0 0 0
> 0 0
> ste3* 1500 <Link#4> 00:0d:88:fc:c8:cf 0 0 0
> 0 0
> bge0 1500 <Link#5> 00:12:79:94:ed:12 11343768 2114510 20564242
> 0 0
> bge1* 1500 <Link#6> 00:12:79:94:ed:11 0 0 0
> 0 0
> pflog 33208 <Link#7> 0 0 0
> 0 0
> lo0 16384 <Link#8> 60221421 0 60221421
> 0 0
> [...]
>
> If you have a patch for FreeBSD 6.2, i'm quite happily to test it on
> our development server.
>
> Regards,
>
> Robin Gruyters
>
Sorry for the late reply - i was AFK for some time and didnt read mails.
Patch against 6.2R attached, please let me know does it helps or not.
--
Oleg.
================================================================
=== Oleg Bulyzhin -- OBUL-RIPN -- OBUL-RIPE -- oleg@rinet.ru ===
================================================================
--k+w/mQv8wyuph6w0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="bge_link.r62.diff"
Index: sys/dev/bge/if_bgereg.h
===================================================================
RCS file: /home/ncvs/src/sys/dev/bge/if_bgereg.h,v
retrieving revision 1.36.2.8.2.1
diff -u -r1.36.2.8.2.1 if_bgereg.h
--- sys/dev/bge/if_bgereg.h 21 Dec 2006 21:53:54 -0000 1.36.2.8.2.1
+++ sys/dev/bge/if_bgereg.h 5 Feb 2007 23:23:22 -0000
@@ -2460,8 +2460,13 @@
uint32_t bge_tx_buf_ratio;
int bge_if_flags;
int bge_txcnt;
- int bge_link; /* link state */
- int bge_link_evt; /* pending link event */
+ uint32_t bge_sts;
+#define BGE_STS_LINK 0x00000001 /* MAC link status */
+#define BGE_STS_LINK_EVT 0x00000002 /* pending link event */
+#define BGE_STS_AUTOPOLL 0x00000004 /* PHY auto-polling */
+#define BGE_STS_BIT(sc, x) ((sc)->bge_sts & (x))
+#define BGE_STS_SETBIT(sc, x) ((sc)->bge_sts |= (x))
+#define BGE_STS_CLRBIT(sc, x) ((sc)->bge_sts &= ~(x))
struct callout bge_stat_ch;
char *bge_vpd_prodname;
char *bge_vpd_readonly;
Index: sys/dev/bge/if_bge.c
===================================================================
RCS file: /home/ncvs/src/sys/dev/bge/if_bge.c,v
retrieving revision 1.91.2.18.2.1
diff -u -r1.91.2.18.2.1 if_bge.c
--- sys/dev/bge/if_bge.c 21 Dec 2006 21:53:54 -0000 1.91.2.18.2.1
+++ sys/dev/bge/if_bge.c 5 Feb 2007 23:23:29 -0000
@@ -602,6 +602,7 @@
/* Reading with autopolling on may trigger PCI errors */
autopoll = CSR_READ_4(sc, BGE_MI_MODE);
if (autopoll & BGE_MIMODE_AUTOPOLL) {
+ BGE_STS_CLRBIT(sc, BGE_STS_AUTOPOLL);
BGE_CLRBIT(sc, BGE_MI_MODE, BGE_MIMODE_AUTOPOLL);
DELAY(40);
}
@@ -615,6 +616,7 @@
}
if (autopoll & BGE_MIMODE_AUTOPOLL) {
+ BGE_STS_SETBIT(sc, BGE_STS_AUTOPOLL);
BGE_SETBIT(sc, BGE_MI_MODE, BGE_MIMODE_AUTOPOLL);
DELAY(40);
}
@@ -1456,6 +1458,7 @@
if (sc->bge_tbi) {
CSR_WRITE_4(sc, BGE_MI_STS, BGE_MISTS_LINK);
} else {
+ BGE_STS_SETBIT(sc, BGE_STS_AUTOPOLL);
BGE_SETBIT(sc, BGE_MI_MODE, BGE_MIMODE_AUTOPOLL|10<<16);
if (sc->bge_asicrev == BGE_ASICREV_BCM5700 &&
sc->bge_chipid != BGE_CHIPID_BCM5700_B2)
@@ -2645,12 +2648,12 @@
/* Note link event. It will be processed by POLL_AND_CHECK_STATUS cmd */
if (statusword & BGE_STATFLAG_LINKSTATE_CHANGED)
- sc->bge_link_evt++;
+ BGE_STS_SETBIT(sc, BGE_STS_LINK_EVT);
if (cmd == POLL_AND_CHECK_STATUS)
if ((sc->bge_asicrev == BGE_ASICREV_BCM5700 &&
sc->bge_chipid != BGE_CHIPID_BCM5700_B2) ||
- sc->bge_link_evt || sc->bge_tbi)
+ BGE_STS_BIT(sc, BGE_STS_LINK_EVT) || sc->bge_tbi)
bge_link_upd(sc);
sc->rxcycles = count;
@@ -2699,7 +2702,7 @@
if ((sc->bge_asicrev == BGE_ASICREV_BCM5700 &&
sc->bge_chipid != BGE_CHIPID_BCM5700_B2) ||
- statusword || sc->bge_link_evt)
+ statusword || BGE_STS_BIT(sc, BGE_STS_LINK_EVT))
bge_link_upd(sc);
if (ifp->if_drv_flags & IFF_DRV_RUNNING) {
@@ -2733,8 +2736,10 @@
bge_stats_update(sc);
if (!sc->bge_tbi) {
- mii = device_get_softc(sc->bge_miibus);
- mii_tick(mii);
+ if (!BGE_STS_BIT(sc, BGE_STS_LINK)) {
+ mii = device_get_softc(sc->bge_miibus);
+ mii_tick(mii);
+ }
} else {
/*
* Since in TBI mode auto-polling can't be used we should poll
@@ -2746,7 +2751,7 @@
if (!(sc->bge_ifp->if_capenable & IFCAP_POLLING))
#endif
{
- sc->bge_link_evt++;
+ BGE_STS_SETBIT(sc, BGE_STS_LINK_EVT);
BGE_SETBIT(sc, BGE_MISC_LOCAL_CTL, BGE_MLC_INTR_SET);
}
}
@@ -2988,7 +2993,7 @@
sc = ifp->if_softc;
- if (!sc->bge_link || IFQ_DRV_IS_EMPTY(&ifp->if_snd))
+ if (!BGE_STS_BIT(sc, BGE_STS_LINK) || IFQ_DRV_IS_EMPTY(&ifp->if_snd))
return;
prodidx = sc->bge_tx_prodidx;
@@ -3264,7 +3269,7 @@
return (0);
}
- sc->bge_link_evt++;
+ BGE_STS_SETBIT(sc, BGE_STS_LINK_EVT);
mii = device_get_softc(sc->bge_miibus);
if (mii->mii_instance) {
struct mii_softc *miisc;
@@ -3563,9 +3568,9 @@
* lead to hardware deadlock. So we just clearing MAC's link state
* (PHY may still have link UP).
*/
- if (bootverbose && sc->bge_link)
+ if (bootverbose && BGE_STS_BIT(sc, BGE_STS_LINK))
if_printf(sc->bge_ifp, "link DOWN\n");
- sc->bge_link = 0;
+ BGE_STS_CLRBIT(sc, BGE_STS_LINK);
ifp->if_drv_flags &= ~(IFF_DRV_RUNNING | IFF_DRV_OACTIVE);
}
@@ -3623,12 +3628,12 @@
bge_link_upd(struct bge_softc *sc)
{
struct mii_data *mii;
- uint32_t link, status;
+ uint32_t status;
BGE_LOCK_ASSERT(sc);
/* Clear 'pending link event' flag. */
- sc->bge_link_evt = 0;
+ BGE_STS_CLRBIT(sc, BGE_STS_LINK_EVT);
/*
* Process link state changes.
@@ -3649,20 +3654,18 @@
sc->bge_chipid != BGE_CHIPID_BCM5700_B2) {
status = CSR_READ_4(sc, BGE_MAC_STS);
if (status & BGE_MACSTAT_MI_INTERRUPT) {
- callout_stop(&sc->bge_stat_ch);
- bge_tick_locked(sc);
-
mii = device_get_softc(sc->bge_miibus);
- if (!sc->bge_link &&
+ mii_pollstat(mii);
+ if (!BGE_STS_BIT(sc, BGE_STS_LINK) &&
mii->mii_media_status & IFM_ACTIVE &&
IFM_SUBTYPE(mii->mii_media_active) != IFM_NONE) {
- sc->bge_link++;
+ BGE_STS_SETBIT(sc, BGE_STS_LINK);
if (bootverbose)
if_printf(sc->bge_ifp, "link UP\n");
- } else if (sc->bge_link &&
+ } else if (BGE_STS_BIT(sc, BGE_STS_LINK) &&
(!(mii->mii_media_status & IFM_ACTIVE) ||
IFM_SUBTYPE(mii->mii_media_active) == IFM_NONE)) {
- sc->bge_link = 0;
+ BGE_STS_CLRBIT(sc, BGE_STS_LINK);
if (bootverbose)
if_printf(sc->bge_ifp, "link DOWN\n");
}
@@ -3680,8 +3683,8 @@
if (sc->bge_tbi) {
status = CSR_READ_4(sc, BGE_MAC_STS);
if (status & BGE_MACSTAT_TBI_PCS_SYNCHED) {
- if (!sc->bge_link) {
- sc->bge_link++;
+ if (!BGE_STS_BIT(sc, BGE_STS_LINK)) {
+ BGE_STS_SETBIT(sc, BGE_STS_LINK);
if (sc->bge_asicrev == BGE_ASICREV_BCM5704)
BGE_CLRBIT(sc, BGE_MAC_MODE,
BGE_MACMODE_TBI_SEND_CFGS);
@@ -3691,42 +3694,38 @@
if_link_state_change(sc->bge_ifp,
LINK_STATE_UP);
}
- } else if (sc->bge_link) {
- sc->bge_link = 0;
+ } else if (BGE_STS_BIT(sc, BGE_STS_LINK)) {
+ BGE_STS_CLRBIT(sc, BGE_STS_LINK);
if (bootverbose)
if_printf(sc->bge_ifp, "link DOWN\n");
if_link_state_change(sc->bge_ifp, LINK_STATE_DOWN);
}
- /* Discard link events for MII/GMII cards if MI auto-polling disabled */
- } else if (CSR_READ_4(sc, BGE_MI_MODE) & BGE_MIMODE_AUTOPOLL) {
- /*
- * Some broken BCM chips have BGE_STATFLAG_LINKSTATE_CHANGED bit
- * in status word always set. Workaround this bug by reading
- * PHY link status directly.
- */
- link = (CSR_READ_4(sc, BGE_MI_STS) & BGE_MISTS_LINK) ? 1 : 0;
-
- if (link != sc->bge_link ||
- sc->bge_asicrev == BGE_ASICREV_BCM5700) {
- callout_stop(&sc->bge_stat_ch);
- bge_tick_locked(sc);
-
- mii = device_get_softc(sc->bge_miibus);
- if (!sc->bge_link &&
- mii->mii_media_status & IFM_ACTIVE &&
- IFM_SUBTYPE(mii->mii_media_active) != IFM_NONE) {
- sc->bge_link++;
- if (bootverbose)
- if_printf(sc->bge_ifp, "link UP\n");
- } else if (sc->bge_link &&
- (!(mii->mii_media_status & IFM_ACTIVE) ||
- IFM_SUBTYPE(mii->mii_media_active) == IFM_NONE)) {
- sc->bge_link = 0;
- if (bootverbose)
- if_printf(sc->bge_ifp, "link DOWN\n");
- }
- }
- }
+ /*
+ * Discard link events for MII/GMII cards if MI auto-polling disabled.
+ * This should not happen since mii callouts are locked now, but
+ * we keep this check for debug.
+ */
+ } else if (BGE_STS_BIT(sc, BGE_STS_AUTOPOLL)) {
+ mii = device_get_softc(sc->bge_miibus);
+ mii_pollstat(mii);
+
+ if (!BGE_STS_BIT(sc, BGE_STS_LINK) &&
+ mii->mii_media_status & IFM_ACTIVE &&
+ IFM_SUBTYPE(mii->mii_media_active) != IFM_NONE) {
+ BGE_STS_SETBIT(sc, BGE_STS_LINK);
+ if (bootverbose)
+ if_printf(sc->bge_ifp, "link UP\n");
+ } else if (BGE_STS_BIT(sc, BGE_STS_LINK) &&
+ (!(mii->mii_media_status & IFM_ACTIVE) ||
+ IFM_SUBTYPE(mii->mii_media_active) == IFM_NONE)) {
+ BGE_STS_CLRBIT(sc, BGE_STS_LINK);
+ if (bootverbose)
+ if_printf(sc->bge_ifp, "link DOWN\n");
+ }
+ } else
+ /* Should not happen, see above. */
+ if_printf(sc->bge_ifp,
+ "link event discarded: PHY auto-polling is off.\n");
/* Clear the attention. */
CSR_WRITE_4(sc, BGE_MAC_STS, BGE_MACSTAT_SYNC_CHANGED|
Index: sys/dev/mii/brgphy.c
===================================================================
RCS file: /home/ncvs/src/sys/dev/mii/brgphy.c,v
retrieving revision 1.34.2.7
diff -u -r1.34.2.7 brgphy.c
--- sys/dev/mii/brgphy.c 21 Oct 2006 00:26:41 -0000 1.34.2.7
+++ sys/dev/mii/brgphy.c 5 Feb 2007 23:23:39 -0000
@@ -464,6 +464,9 @@
return;
}
+ if (bmsr & BRGPHY_BMSR_LINK)
+ sc->mii_ticks = 0; /* Reset autoneg timer. */
+
switch (PHY_READ(sc, BRGPHY_MII_AUXSTS) &
BRGPHY_AUXSTS_AN_RES) {
case BRGPHY_RES_1000FD:
--k+w/mQv8wyuph6w0--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070205232800.GA45487>