Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 8 Jan 2015 13:40:11 +0100
From:      Dimitry Andric <dim@FreeBSD.org>
To:        "Bjoern A. Zeeb" <bzeeb-lists@lists.zabbadoz.net>
Cc:        Ed Maste <emaste@freebsd.org>, "current@freebsd.org" <current@freebsd.org>
Subject:   Re: "*** [kernel.debug] Error code 139"?
Message-ID:  <A01756B6-1B4E-48D2-BFF9-B7B29F4AA337@FreeBSD.org>
In-Reply-To: <3C59182B-D747-4F33-9AED-6DED638596E7@lists.zabbadoz.net>
References:  <20150107135756.GC14822@albert.catwhisker.org> <CAPyFy2CHDc%2BskpJzrVgborDYgUK3CcPVcGeJiKOfo9UqnDfrfw@mail.gmail.com> <A27C6742-BCD7-43E5-9ED1-D34A6BF0C8E4@lists.zabbadoz.net> <3C59182B-D747-4F33-9AED-6DED638596E7@lists.zabbadoz.net>

next in thread | previous in thread | raw e-mail | index | archive | help

--Apple-Mail=_B943B119-C438-4949-85BB-9D3AAECA2963
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

On 08 Jan 2015, at 13:09, Bjoern A. Zeeb =
<bzeeb-lists@lists.zabbadoz.net> wrote:
>=20
>> On 07 Jan 2015, at 15:02 , Bjoern A. Zeeb =
<bzeeb-lists@lists.zabbadoz.net> wrote:
>>> On 07 Jan 2015, at 14:13 , Ed Maste <emaste@freebsd.org> wrote:
>>> On 7 January 2015 at 08:57, David Wolfskill <david@catwhisker.org> =
wrote:
>>>> --- kernel.debug ---
>>>> linking kernel.debug
>>>> ctfmerge -L VERSION -g -o kernel.debug ...
>>>> Segmentation fault (core dumped)
>>>> *** [kernel.debug] Error code 139
>>> ...
>>>> I'm mostly wondering what "Error code 139" is intended to mean.... =
:-}
>>>=20
>>> It's the way termination due to a signal is reported. The exit =
status
>>> is 128 + the signal number and SIGSEGV =3D 11.
>>=20
>> I did see it twice last night while doing HEAD builds, and =
i386.GENERIC only.  For later builds it was just gone again.  Not sure =
if anyone wants to figure out what was special about the two SVN =
revisions.   For me it was a cross-build from amd64 as part of make =
universe.
>=20
> Ok, this continues;  I have since seen it four more times.  Still =
i386.GENERIC only; none of the LINT kernels or other architectures.
>=20
> Can someone please investigate what triggers this?

Looks like a stack overflow, cause unknown as of yet:

Core was generated by `ctfmerge'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0804e7ac in hash_match (hash=3D0x2cd0e460, key=3D0xbb6f9048, =
private=3D<optimized out>, fun=3D<optimized out>)
    at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/hash.c:147
147		int bucket =3D hash->h_hashfn(hash->h_nbuckets, key);
(gdb) bt
#0  0x0804e7ac in hash_match (hash=3D0x2cd0e460, key=3D0xbb6f9048, =
private=3D<optimized out>, fun=3D<optimized out>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/hash.c:147
#1  hash_find (hash=3D0x804e80d <hash_find_first_cb+29>, key=3D0xbb6f9088,=
 value=3D0x3e5) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/hash.c:207
#2  0x08049e60 in alist_find (alist=3D0xbb6f9044, name=3D0x804e7f0 =
<hash_find_first_cb>, value=3D0x3e5) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/alist.c:130
#3  0x080508e6 in get_mapping (ta=3D0xbb6f9048, srcid=3D997) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:195
#4  equiv_node (ctdp=3D0x293f2730, mtdp=3D0x3207e2b0, ed=3D0xbb8faa58) =
at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:366
#5  0x080504b5 in equiv_su (stdp=3D0x2ecb0970, ttdp=3D0x3207e2e0, =
ed=3D<optimized out>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:292
#6  0x0805094f in equiv_node (ctdp=3D0x2ecb0970, mtdp=3D0x3207e2e0, =
ed=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:385
#7  0x0805013b in equiv_plain (stdp=3D0x293f2730, ttdp=3D0x3207e2b0, =
ed=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:243
#8  0x0805094f in equiv_node (ctdp=3D0x293f2730, mtdp=3D0x3207e2b0, =
ed=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:385
#9  0x080504b5 in equiv_su (stdp=3D0x2ecb0970, ttdp=3D0x3207e2e0, =
ed=3D<optimized out>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:292
#10 0x0805094f in equiv_node (ctdp=3D0x2ecb0970, mtdp=3D0x3207e2e0, =
ed=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:385
#11 0x0805013b in equiv_plain (stdp=3D0x293f2730, ttdp=3D0x3207e2b0, =
ed=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:243
#12 0x0805094f in equiv_node (ctdp=3D0x293f2730, mtdp=3D0x3207e2b0, =
ed=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:385
#13 0x080504b5 in equiv_su (stdp=3D0x2ecb0970, ttdp=3D0x3207e2e0, =
ed=3D<optimized out>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:292
#14 0x0805094f in equiv_node (ctdp=3D0x2ecb0970, mtdp=3D0x3207e2e0, =
ed=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:385
[...rougly 75000 instances of the same...]
#75124 0x0805094f in equiv_node (ctdp=3D0x293f2730, mtdp=3D0x3207e2b0, =
ed=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:385
#75125 0x080504b5 in equiv_su (stdp=3D0x293ab2e0, ttdp=3D0x32066c40, =
ed=3D<optimized out>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:292
#75126 0x0805094f in equiv_node (ctdp=3D0x293ab2e0, mtdp=3D0x32066c40, =
ed=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:385
#75127 0x0805013b in equiv_plain (stdp=3D0x293ab2b0, ttdp=3D0x32066c10, =
ed=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:243
#75128 0x0805094f in equiv_node (ctdp=3D0x293ab2b0, mtdp=3D0x32066c10, =
ed=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:385
#75129 0x080504b5 in equiv_su (stdp=3D0x2ecb0e20, ttdp=3D0x3208f9d0, =
ed=3D<optimized out>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:292
#75130 0x0805094f in equiv_node (ctdp=3D0x2ecb0e20, mtdp=3D0x3208f9d0, =
ed=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:385
#75131 0x0805013b in equiv_plain (stdp=3D0x2ecb0df0, ttdp=3D0x3208f9a0, =
ed=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:243
#75132 0x0805094f in equiv_node (ctdp=3D0x2ecb0df0, mtdp=3D0x3208f9a0, =
ed=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:385
#75133 0x0805121d in equiv_cb (bucket=3D0x3208f9a0, arg=3D0xbb8faa58) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:412
#75134 0x0804e772 in hash_find_list_cb (node=3D0x3208f9a0, =
arg=3D0xbb8faa10) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/hash.c:160
#75135 0x0804fa31 in list_iter (list=3D0x3208bb18, func=3D0x804e740 =
<hash_find_list_cb>, private=3D<optimized out>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/common/list.c:127
#75136 0x0804e735 in hash_find_iter (hash=3D<optimized out>, =
key=3D<optimized out>, fun=3D<optimized out>, private=3D<optimized out>) =
at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/hash.c:180
#75137 0x080510c1 in map_td_tree_post (ctdp=3D0x2ecb0df0, =
ctdpp=3D<optimized out>, private=3D0xbb8faf08) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:450
#75138 0x08053603 in tdtraverse (this=3D0x2ecb0df0, thisp=3D0x29268f88, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:180
#75139 0x080538bb in tdtrav_func (this=3D0x2ecb0dc0, tdtd=3D<optimized =
out>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:70
#75140 0x080535e2 in tdtraverse (this=3D0x2ecb0dc0, thisp=3D0x2ecb0da8, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75141 0x08053802 in tdtrav_plain (this=3D0x2ecb0d90, tdtd=3D0xbb8fae40) =
at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:57
#75142 0x080535e2 in tdtraverse (this=3D0x2ecb0d90, thisp=3D0x2ecb0d78, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75143 0x08053802 in tdtrav_plain (this=3D0x2ecb0d60, tdtd=3D0xbb8fae40) =
at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:57
#75144 0x080535e2 in tdtraverse (this=3D0x2ecb0d60, thisp=3D0x2912822c, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75145 0x08053916 in tdtrav_su (this=3D0x2ecb0970, tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:98
#75146 0x080535e2 in tdtraverse (this=3D0x2ecb0970, thisp=3D0x293f2748, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75147 0x08053802 in tdtrav_plain (this=3D0x293f2730, tdtd=3D0xbb8fae40) =
at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:57
#75148 0x080535e2 in tdtraverse (this=3D0x293f2730, thisp=3D0x29098aec, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75149 0x08053916 in tdtrav_su (this=3D0x293ab2e0, tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:98
#75150 0x080535e2 in tdtraverse (this=3D0x293ab2e0, thisp=3D0x293ab2c8, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75151 0x08053802 in tdtrav_plain (this=3D0x293ab2b0, tdtd=3D0xbb8fae40) =
at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:57
#75152 0x080535e2 in tdtraverse (this=3D0x293ab2b0, thisp=3D0x2d44c298, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75153 0x08053802 in tdtrav_plain (this=3D0x2d44c280, tdtd=3D0xbb8fae40) =
at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:57
#75154 0x080535e2 in tdtraverse (this=3D0x2d44c280, thisp=3D0x2d4b48d8, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75155 0x080538bb in tdtrav_func (this=3D0x2937bd30, tdtd=3D<optimized =
out>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:70
#75156 0x080535e2 in tdtraverse (this=3D0x2937bd30, thisp=3D0x2937bd18, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75157 0x08053802 in tdtrav_plain (this=3D0x2937bd00, tdtd=3D0xbb8fae40) =
at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:57
#75158 0x080535e2 in tdtraverse (this=3D0x2937bd00, thisp=3D0x2937bce8, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75159 0x08053802 in tdtrav_plain (this=3D0x2937bcd0, tdtd=3D0xbb8fae40) =
at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:57
#75160 0x080535e2 in tdtraverse (this=3D0x2937bcd0, thisp=3D0x2909874c, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75161 0x08053916 in tdtrav_su (this=3D0x29374250, tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:98
#75162 0x080535e2 in tdtraverse (this=3D0x29374250, thisp=3D0x29374238, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75163 0x08053802 in tdtrav_plain (this=3D0x29374220, tdtd=3D0xbb8fae40) =
at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:57
#75164 0x080535e2 in tdtraverse (this=3D0x29374220, thisp=3D0x290982ac, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75165 0x08053916 in tdtrav_su (this=3D0x293741c0, tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:98
#75166 0x080535e2 in tdtraverse (this=3D0x293741c0, thisp=3D0x293741a8, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75167 0x08053802 in tdtrav_plain (this=3D0x29374190, tdtd=3D0xbb8fae40) =
at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:57
#75168 0x080535e2 in tdtraverse (this=3D0x29374190, thisp=3D0x2d61cd4c, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75169 0x08053916 in tdtrav_su (this=3D0x29374130, tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:98
#75170 0x080535e2 in tdtraverse (this=3D0x29374130, thisp=3D0x2ef762c8, =
tdtd=3D0xbb8fae40) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:176
#75171 0x0805370f in iitraverse_td (arg1=3D<optimized out>, =
arg2=3D<optimized out>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:194
#75172 iitraverse (ii=3D0x2ef762c0, vgenp=3D0x8050ed0 <iidesc_match>, =
firstops=3D0x895, preops=3D0x805697c <tdnops+24>, postops=3D0x80563f0 =
<map_pre>, private=3D0x80563b8 <map_post>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/traverse.c:214
#75173 0x080509cf in merge_type_cb (data=3D0x2ef762c0, arg=3D0xbb8faf08) =
at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:657
#75174 0x0804fa31 in list_iter (list=3D0x2926e8d0, func=3D0x8050980 =
<merge_type_cb>, private=3D<optimized out>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/common/list.c:127
#75175 0x0804e5e0 in hash_iter (hash=3D0x2b389f20, fun=3D<optimized =
out>, private=3D<optimized out>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/hash.c:223
#75176 0x0804fdf1 in merge_types (src=3D0x2b389f20, mcd=3D0x2dd522e0) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:1045
#75177 merge_into_master (cur=3D<optimized out>, mstr=3D0x2dd522e0, =
tgt=3D<optimized out>, selfuniquify=3D<optimized out>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/merge.c:1102
#75178 0x0804e0f7 in worker_runphase2 (wq=3D0x80568f0 <wq+24>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/ctfmerge.c:489
#75179 worker_thread (wq=3D0x80568f0 <wq+24>) at =
/usr/src/cddl/usr.bin/ctfmerge/../../../cddl/contrib/opensolaris/tools/ctf=
/cvt/ctfmerge.c:542
#75180 0x280e869c in ?? () from /lib/libthr.so.3
#75181 0x00000000 in ?? ()

-Dimitry


--Apple-Mail=_B943B119-C438-4949-85BB-9D3AAECA2963
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename=signature.asc
Content-Type: application/pgp-signature;
	name=signature.asc
Content-Description: Message signed with OpenPGP using GPGMail

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.26

iEYEARECAAYFAlSuerMACgkQsF6jCi4glqMBxgCgpQZNtHpmFW2EuDCdUeWUvbYA
I74AnR7wMFUn5NyjL1VHNImzrCfb9TX8
=pdjp
-----END PGP SIGNATURE-----

--Apple-Mail=_B943B119-C438-4949-85BB-9D3AAECA2963--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A01756B6-1B4E-48D2-BFF9-B7B29F4AA337>