Date: Mon, 02 May 2022 08:25:39 -0700 From: Chris <bsd-lists@bsdforge.com> To: freebsd-net@freebsd.org, freebsd-arm@freebsd.org Subject: Re: 60+% ping packet loss on Pi3 under -current and stable-13 Message-ID: <500d4bb5b672b8c3fac18bf76f781230@bsdforge.com> In-Reply-To: <CA5CAD79-F2B6-464D-978A-B9FFC8D2125A@yahoo.com> References: <D0DFF445-9B68-4C64-B9CA-D68149ACD89E@yahoo.com> <20220314010330.GA70447@www.zefox.net> <9FA3F874-2987-44A2-A987-F905E78CCA65@yahoo.com> <20220428023226.GA5666@www.zefox.net> <8A57B411-AA06-47F4-935D-EDF45D8DF0EC@yahoo.com> <70C2DF4B-D08B-491D-B7B5-1EAD0D1BF0E3@yahoo.com> <20220429005206.GA1171@www.zefox.net> <A5172CAE-7327-4B13-94F7-989607D01237@yahoo.com> <20220430021207.GA7600@www.zefox.net> <FE6B7DD6-4F71-430C-A78D-480D18F45CCA@iitbombay.org> <20220501181254.GA14961@www.zefox.net> <E0424240-1E6A-446F-A5BC-939BE8D5A6F2@yahoo.com> <CA5CAD79-F2B6-464D-978A-B9FFC8D2125A@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--=_561093c953b62cf25d5e022249f2d35a
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII;
format=flowed
On 2022-05-01 12:58, Mark Millard wrote:
> On 2022-May-1, at 12:15, Mark Millard <marklmi@yahoo.com> wrote:
>
>
>> On 2022-May-1, at 11:12, bob prohaska <fbsd@www.zefox.net> wrote:
>>
>>> On Sat, Apr 30, 2022 at 06:39:57PM -0700, Bakul Shah wrote:
>>>> On Apr 29, 2022, at 7:12 PM, bob prohaska <fbsd@www.zefox.net> wrote:
>>>>>
>>>>> Since about December of 2021 I've been noticing problems with
>>>>> wired network connectivity on a pair of raspberry pi 3 machines
>>>>> using wired network connections. One runs stable-13.1, the other
>>>>> runs -current, both are up to date as of a few days ago.
>>>>>
>>>>> Essentially both machines fail to respond to inbound network
>>>>> connections via ssh or ping after reboot. If I get on the
>>>>> serial console and start an outbound ping to anywhere, both
>>>>> machines respond to incoming pings with about a 65% packet
>>>>> loss.
>>>
>>>> Suggest running tcpdump on the rpi3 to see what is going on
>>>> when connected to the public vs private net.
>>>>
>>>
>>> Public net first, since that's where the machine is now. Gateway.zefox.net
>>> is the name of my router's public interface, dcn.org belongs to my isp and
>>> fusionbroadband is their service provider..
>>>
>>> While on the -current Pi3 serial console (with no outbound ping running)
>>> and no inbound traffic from my hosts I see after a couple minutes:
>>>
>>> root@www:/mnt # tcpdump
>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>>> listening on ue0, link-type EN10MB (Ethernet), capture size 262144 bytes
>>> 10:39:40.887853 ARP, Request who-has www.zefox.org tell gateway.zefox.net,
>>> length 46
>>> 10:39:40.887929 ARP, Reply www.zefox.org is-at b8:27:eb:71:46:4e (oui
>>> Unknown), length 28
>>> 10:39:40.893220 ARP, Request who-has
>>> 50-1-20-1.dsl.static.fusionbroadband.com tell www.zefox.org, length 28
>>> 10:39:40.915469 ARP, Reply 50-1-20-1.dsl.static.fusionbroadband.com is-at
>>> 00:1b:90:d2:4a:c4 (oui Unknown), length 50
>>> 10:39:40.915529 IP www.zefox.org.50714 > spoke.dcn.davis.ca.us.domain:
>>> 51409+ PTR? 28.20.1.50.in-addr.arpa. (41)
>>> 10:39:40.943602 IP spoke.dcn.davis.ca.us.domain > www.zefox.org.50714:
>>> 51409 1/3/6 PTR www.zefox.org. (265)
>>> 10:39:40.945416 IP www.zefox.org.15986 > spoke.dcn.davis.ca.us.domain:
>>> 44966+ PTR? 31.20.1.50.in-addr.arpa. (41)
>>> 10:39:40.973487 IP spoke.dcn.davis.ca.us.domain > www.zefox.org.15986:
>>> 44966 1/3/6 PTR gateway.zefox.net. (266)
>>> 10:39:40.975037 IP www.zefox.org.57611 > spoke.dcn.davis.ca.us.domain:
>>> 31749+ PTR? 1.20.1.50.in-addr.arpa. (40)
>>> 10:39:46.288219 IP www.zefox.org.49710 > wheel.dcn.davis.ca.us.domain:
>>> 31749+ PTR? 1.20.1.50.in-addr.arpa. (40)
>>> 10:39:46.316239 IP wheel.dcn.davis.ca.us.domain > www.zefox.org.49710:
>>> 31749 1/3/6 PTR 50-1-20-1.dsl.static.fusionbroadband.com. (291)
>>> 10:39:46.318267 IP www.zefox.org.17061 > spoke.dcn.davis.ca.us.domain:
>>> 37579+ PTR? 2.253.150.168.in-addr.arpa. (44)
>>> 10:39:46.346851 IP spoke.dcn.davis.ca.us.domain > www.zefox.org.17061:
>>> 37579* 1/2/2 PTR spoke.dcn.davis.ca.us. (145)
>>> 10:39:46.348674 IP www.zefox.org.40440 > spoke.dcn.davis.ca.us.domain:
>>> 20572+ PTR? 1.253.150.168.in-addr.arpa. (44)
>>> 10:39:51.420705 IP www.zefox.org.64019 > wheel.dcn.davis.ca.us.domain:
>>> 20572+ PTR? 1.253.150.168.in-addr.arpa. (44)
>>> 10:39:51.448850 IP wheel.dcn.davis.ca.us.domain > www.zefox.org.64019:
>>> 20572* 1/2/2 PTR wheel.dcn.davis.ca.us. (145)
>>> 10:40:40.147603 ARP, Request who-has
>>> 50-1-20-1.dsl.static.fusionbroadband.com tell ns1.zefox.net, length 46
>>> 10:40:40.148844 IP www.zefox.org.46127 > spoke.dcn.davis.ca.us.domain:
>>> 12186+ PTR? 29.20.1.50.in-addr.arpa. (41)
>>> 10:40:40.176486 IP spoke.dcn.davis.ca.us.domain > www.zefox.org.46127:
>>> 12186 1/3/6 PTR ns1.zefox.net. (262)
>>> 10:40:57.688225 ARP, Request who-has www.zefox.org tell gateway.zefox.net,
>>> length 46
>>> 10:40:57.688305 ARP, Reply www.zefox.org is-at b8:27:eb:71:46:4e (oui
>>> Unknown), length 28
>>> 10:42:14.488727 ARP, Request who-has www.zefox.org tell gateway.zefox.net,
>>> length 46
>>> 10:42:14.488804 ARP, Reply www.zefox.org is-at b8:27:eb:71:46:4e (oui
>>> Unknown), length 28
>>> 10:42:43.761226 ARP, Request who-has
>>> 50-1-20-1.dsl.static.fusionbroadband.com tell www.zefox.com, length 46
>>> 10:42:43.762522 IP www.zefox.org.56181 > spoke.dcn.davis.ca.us.domain:
>>> 28779+ PTR? 26.20.1.50.in-addr.arpa. (41)
>>> 10:42:43.790361 IP spoke.dcn.davis.ca.us.domain > www.zefox.org.56181:
>>> 28779 1/3/6 PTR www.zefox.com. (265)
>>> 10:43:31.289103 ARP, Request who-has www.zefox.org tell gateway.zefox.net,
>>> length 46
>>> 10:43:31.289181 ARP, Reply www.zefox.org is-at b8:27:eb:71:46:4e (oui
>>> Unknown), length 28
>>>
>>> If I now start an inbound ping from one of my hosts it gets no reply and
>>> tcpdump reports no additional traffic. With an outbound ping running
>>> there's
>>> at least a sparse reply.
>>>
>>> ^C
>>> 28 packets captured
>>> 28 packets received by filter
>>> 0 packets dropped by kernel
>>> root@www:/mnt #
>>>
>>> The "oui unknown" looks like some sort of failure.....
>>> Can you ping www.zefox.org? I have no outside vantage point.
>>> There is still no outbound ping running and I would expect
>>> you'll get no or very sparse reply.
>>>
>>>
>>> Thus far only the two Pi3s suffer from connectivity problems; Pi2s and a
>>> Pi4 have
>>> no difficulty on the same address block. Is there a switch for tcpdump
>>> that will
>>> limit records to relevant traffic? Otherwise it's a flood.
>>>
>>> These results were obtained after standing idle overnight and
>>> are rather different (in ways I don't understand) from behavior
>>> immediately after reboot, I'll have to repeat as I learn more.
>>
>> I wonder if there is a notable difference between
>> monitoring traffic from 2 places:
>>
>> A) from the machine seeing the problem
>> vs.
>> B) from a machine not having problems but
>> connected were all the traffic would be
>> on the wire it is connected to.
>>
>> It may be that monitoring from both and
>> comparing/contrasting the reported traffic
>> from the two provides additional evidence.
>>
>> There may be modes of monitoring that are
>> relevant for this. But I'm not familiar
>> with any detail here.
>>
>>
>> For reference:
>>
>> # ping www.zefox.org
>> PING www.zefox.org (50.1.20.28): 56 data bytes
>> ^C
>> --- www.zefox.org ping statistics ---
>> 32 packets transmitted, 0 packets received, 100.0% packet loss
>>
>> I found the command traceroute and it reports:
>>
>> # traceroute www.zefox.org
>> traceroute to www.zefox.org (50.1.20.28), 64 hops max, 40 byte packets
>> 1 192.168.1.1 (192.168.1.1) 0.697 ms 0.486 ms 1.277 ms
>> 2 172.30.26.66 (172.30.26.66) 30.019 ms
>> 172.30.26.67 (172.30.26.67) 41.720 ms
>> 172.30.26.66 (172.30.26.66) 28.645 ms
>> 3 68.85.243.125 (68.85.243.125) 8.967 ms
>> 68.85.243.77 (68.85.243.77) 11.462 ms
>> 68.85.243.125 (68.85.243.125) 10.254 ms
>> 4 24.124.129.106 (24.124.129.106) 7.510 ms
>> 96.216.60.165 (96.216.60.165) 10.176 ms
>> 24.124.129.106 (24.124.129.106) 8.945 ms
>> 5 68.85.243.197 (68.85.243.197) 10.837 ms
>> 96.216.60.165 (96.216.60.165) 10.252 ms
>> 68.85.243.197 (68.85.243.197) 16.036 ms
>> 6 68.85.243.197 (68.85.243.197) 14.660 ms
>> be-36211-cs01.seattle.wa.ibone.comcast.net (68.86.93.49) 14.629 ms
>> 68.85.243.197 (68.85.243.197) 8.849 ms
>> 7 be-2412-pe12.seattle.wa.ibone.comcast.net (96.110.34.142) 14.607 ms
>> be-36221-cs02.seattle.wa.ibone.comcast.net (68.86.93.53) 14.122 ms
>> be-2212-pe12.seattle.wa.ibone.comcast.net (96.110.34.134) 13.877 ms
>> 8 be-2412-pe12.seattle.wa.ibone.comcast.net (96.110.34.142) 14.133 ms *
>> 13.663 ms
>> 9 be2075.ccr21.sfo01.atlas.cogentco.com (154.54.0.233) 30.176 ms *
>> be3717.ccr22.sfo01.atlas.cogentco.com (154.54.86.209) 29.002 ms
>> 10 be3717.ccr22.sfo01.atlas.cogentco.com (154.54.86.209) 28.477 ms
>> be2430.ccr31.sjc04.atlas.cogentco.com (154.54.88.186) 27.203 ms
>> be2075.ccr21.sfo01.atlas.cogentco.com (154.54.0.233) 28.515 ms
>> 11 38.104.141.82 (38.104.141.82) 29.820 ms
>> be2430.ccr31.sjc04.atlas.cogentco.com (154.54.88.186) 28.605 ms
>> 38.104.141.82 (38.104.141.82) 33.735 ms
>> 12 38.104.141.82 (38.104.141.82) 27.160 ms
>> 0.xe-0-3-0.scrm-gw1.scrmca01.sonic.net (135.180.179.146) 32.336 ms
>> 38.104.141.82 (38.104.141.82) 31.867 ms
>> 13 0.xe-0-0-0.cr1.scrmca13.sonic.net (135.180.179.166) 31.761 ms
>> 0.xe-0-3-0.scrm-gw1.scrmca01.sonic.net (135.180.179.146) 29.864 ms
>> 0.xe-0-0-0.cr1.scrmca13.sonic.net (135.180.179.166) 31.711 ms
>> 14 0.xe-0-0-0.cr1.scrmca13.sonic.net (135.180.179.166) 30.373 ms
>> gig1-1-1.gw.wscrca11.sonic.net (50.1.36.106) 35.567 ms
>> 0.xe-0-0-0.cr1.scrmca13.sonic.net (135.180.179.166) 31.146 ms
>> 15 gig1-1-1.gw.davsca11.sonic.net (50.1.36.110) 31.513 ms
>> gig1-1-1.gw.wscrca11.sonic.net (50.1.36.106) 31.203 ms
>> gig1-1-1.gw.davsca11.sonic.net (50.1.36.110) 31.354 ms
>> 16 gig1-1-1.gw.davsca11.sonic.net (50.1.36.110) 30.125 ms * 31.996 ms
>> 17 * * *
>> 18 * * *
>> 19 * * *
>> 20 * * *
>> 21 * * *
>> 22 * * *
>> 23 * * *
>> 24 * * *
>> 25 * * *
>> 26 * * *
>> 27 * * *
>> 28 * * *
>> 29 * * *
>> 30 * * *
>> ^C
>>
>> (There did not seem to be much point in having it continue.)
>
> I found and built a port called net/mtr-nox11
> ("My traceroute") and tried it, letting it just
> run. The initial try eventually got a connection
> but reported a 99.2% packet loss as of when I
> captured the below:
>
> My traceroute [v0.95]
> amd64_ZFS (192.168.1.120) -> www.zefox.org (50.1.20.28)
> 2022-05-01T12:40:22-0700
> Keys: Help Display mode Restart statistics Order of fields quit
> Packets Pings
> Host Loss% Snt Last Avg
> Best Wrst StDev
> 1. 192.168.1.1 0.0% 135 0.4 0.8
> 0.1 3.1 0.4
> 2. 172.30.26.66 0.0% 134 28.2 26.1
> 9.3 132.7 18.1
> 3. 68.85.243.77 0.0% 134 8.6 9.0
> 7.5 11.2 0.8
> 4. 24.124.129.106 0.0% 134 10.2 9.1
> 7.6 13.4 0.9
> 5. 96.216.60.165 0.0% 134 9.0 9.1
> 7.8 14.3 0.9
> 6. 68.85.243.197 0.0% 134 14.4 13.6
> 9.2 44.3 5.4
> 7. be-36241-cs04.seattle.wa.ibone.comcast.net 0.0% 134 16.8 14.9
> 13.0 22.6 1.1
> 8. be-2412-pe12.seattle.wa.ibone.comcast.net 0.0% 134 13.5 15.0
> 12.8 46.4 3.2
> 9. (waiting for reply)
> 10. be2075.ccr21.sfo01.atlas.cogentco.com 0.0% 134 29.3 29.0
> 26.7 54.1 2.9
> 11. be2379.ccr31.sjc04.atlas.cogentco.com 0.0% 134 28.0 28.7
> 27.1 40.3 1.3
> 12. 38.104.141.82 0.0% 134 28.0 33.8
> 26.6 114.8 16.5
> 13. 0.xe-0-3-0.scrm-gw1.scrmca01.sonic.net 0.0% 134 30.9 31.0
> 29.0 33.7 0.8
> 14. 0.xe-0-0-0.cr1.scrmca13.sonic.net 0.0% 134 31.1 32.3
> 29.3 93.2 6.7
> 15. gig1-1-1.gw.wscrca11.sonic.net 0.0% 134 31.3 34.9
> 29.5 330.4 26.5
> 16. gig1-1-1.gw.davsca11.sonic.net 0.0% 134 32.8 32.1
> 29.9 44.1 1.7
> 17. (waiting for reply)
> 18. (waiting for reply)
> 19. www.zefox.org 99.2% 134 74.9 74.9
> 74.9 74.9 0.0
>
> I stopped and restarted it and so far no connection
> -- waiting even longer than that first time: Snt
> is now over 600. Rows 18 and 19 have not shown up,
> the last is 17.
>
> . . . (some more time goes by) . . .
>
> I have now stopped it, avoiding the extra load on the
> machines and network.
>
> Looks like there is some problem getting past
> gig1-1-1.gw.davsca11.sonic.net .
Apologies in advance if I'm just making noise. But here's what I see
on a 10Gb network attempting the same traceroute(8)
# traceroute www.zefox.org
traceroute to www.zefox.org (50.1.20.28), 64 hops max, 40 byte packets
1 static-24-113-41-1.wavecable.com (24.113.41.1) 19.918 ms 16.258 ms
13.852 ms
2 174.127.183.72 (174.127.183.72) 18.036 ms 19.647 ms 18.428 ms
3 be4.cr2-sea-b.bb.as11404.net (174.127.137.16) 16.318 ms 19.963 ms
22.306 ms
4 be1.cr2-sea-a.bb.as11404.net (174.127.149.136) 19.391 ms 14.457 ms
15.808 ms
5 sea-b2-link.ip.twelve99.net (62.115.49.138) 19.613 ms 22.770 ms 20.330
ms
6 sjo-b23-link.ip.twelve99.net (62.115.118.169) 39.478 ms 32.428 ms
34.416 ms
7 palo-b24-link.ip.twelve99.net (62.115.115.216) 70.207 ms 41.846 ms
37.838 ms
8 sonicnet-ic350733-palo-b24.ip.twelve99-cust.net (62.115.181.227) 44.718
ms 33.959 ms 42.723 ms
9 0.xe-0-3-0.scrm-gw1.scrmca01.sonic.net (135.180.179.146) 41.699 ms
42.660 ms 114.578 ms
10 0.xe-0-0-0.cr1.scrmca13.sonic.net (135.180.179.166) 47.851 ms 51.590 ms
41.286 ms
11 gig1-1-1.gw.wscrca11.sonic.net (50.1.36.106) 51.199 ms 39.567 ms
40.553 ms
12 gig1-1-1.gw.davsca11.sonic.net (50.1.36.110) 45.005 ms 44.096 ms
41.183 ms
13 * * *
14 * www.zefox.org (50.1.20.28) 62.422 ms *
A trip to sonic net indicates they brag on having better privacy than
their competition. Are they using any privacy extensions that may affect
your ability to ping(8) || traceroute(8) -- TCP/UDP/ICMP? Or is it just
that gig1-1-1.gw.davsca11.sonic.net's BGP is out of date (stale)?
HTH
--Chris
>
> ===
> Mark Millard
> marklmi at yahoo.com
--=_561093c953b62cf25d5e022249f2d35a
Content-Transfer-Encoding: 7bit
Content-Type: application/pgp-keys;
name=0xBDE49540.asc
Content-Disposition: attachment;
filename=0xBDE49540.asc;
size=5028
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQENBGDTzGEBCADHlXdS4V57s2soaEK2wi3o9rr9zo7to/giBSxCpFYJxOnPkL5A
2ibbvflrL8sWvAczx47wgDS7iIhzICBBRdnXtcFGnoeeriV27LSn+PcpnIB+DaWZ
xe+6TDC0Z0JUJ7qDTjUBFzhnQGYlrVvc4WbnWTjJaB1LEwgIX8JqX5S3SX0/oXgs
+OtqDuENZ4/a5te5xPnspTv/5NJHjqYGxjHP0Vw0KjRKS1AoJ1SBPSMQV5373AX9
5NzFS+CjqeQhjfHFPeRajQ8t4T6eqhKA7LtKMO1egeAwNehk9ZoEqEBT2+ojuKUd
oSuzqvhhx+eUIYLFqoPSzMKR+YbStzergsbnABEBAAG0KUNocmlzIEh1dGNoaW5z
b24gPGNocmlzaEB1bHRpbWF0ZWRucy5uZXQ+iQFrBBABCABVBgsJBwgDAgQVCAoC
AxYCAQIZAQIbAwIeARgYaGtwczovL2tleXMub3BlbnBncC5vcmcWIQQGJAsyyBlk
cuwsSYsYdR58veSVQAUCYNQl+wUJA8LAmgAKCRAYdR58veSVQN3NB/sFTeXrZeDk
ml/dshET8QbkOPgXlnibk8+Mauf+y9LjS9WT7R8EmqhK7T7aw115JQ1RWTM6kpQM
jyDBjYF7piJEpNKI9YDeSnODKir1fWQqm9+wd68wAKGvV4m8kg9uOHCvXG4J++MG
zDFH+PuGVxKirFnaz46DpS0Zw7wTtjNiNFvCooYov3IeYGfqcchd3hwBuXgWLexZ
vI8JW7lL9oXl7B/wcbSxg9rwy6/QLYGg6sEtYRcFYyvQWefSMJaLWjU/pZN2iSxM
lXm55iZv1BXHupfeD1ldRiGs6ejrcpa8+U1ju291WbLzcIsU8IDljeW9/WB2dLFT
hJmY1wRk158AtB5DaHJpcyA8YnNkLWxpc3RzQGJzZGZvcmdlLmNvbT6JAWgEEAEI
AFIGCwkHCAMCBBUICgIDFgIBAhsDAh4BGBhoa3BzOi8va2V5cy5vcGVucGdwLm9y
ZxYhBAYkCzLIGWRy7CxJixh1Hny95JVABQJg1CX7BQkDwsCaAAoJEBh1Hny95JVA
aI0H/AlJAOfc5TcMKa479Itw31mwccKb+u0DPN9Gkm/RfWIBjeqqozxCM8G8jVFr
dt/J6KmBO3dQtRZHlXdD57RAfDDl5Vm3uws0s+UIFOxMiua/YxyuDcKLsE8Bjkzx
z+vuJ8f6cg4WlygPr3bo3l81AOuU/wOsTrNkQvVJxgATlooATSVxs0yNn2uoso9f
nhMGUYsmT4c35JYh0k6Lq7Z2LS+ELipMTQ7M7iCWSP1O/zSEvPD4NBo52xCvjLka
KcL4fRl7UN+6ouwGr5aUn83tztE/IR0AK45gFvL5yxI4g/zm1t3j2+hhhW1pBU8w
uQWkD2DyLTWy7xs1uVF5m1ojHp60H0NocmlzIDxrbm90QHRhY29tYXdpcmVsZXNz
Lm5ldD6JAWgEEAEIAFIGCwkHCAMCBBUICgIDFgIBAhsDAh4BGBhoa3BzOi8va2V5
cy5vcGVucGdwLm9yZxYhBAYkCzLIGWRy7CxJixh1Hny95JVABQJg1CX7BQkDwsCa
AAoJEBh1Hny95JVA5m8H/iENaTD4j5QHfaHfiDIdxGx36GnETyRK0vAzr2b6pzG+
7VHNCm4ZfuMsXDJ1ZD8fjTipvg0f4w31xCQI0NgNdAqudBqE075Jwcr9pE9j8VN1
Nvejto01cgLHODbLPhokrkFz1K023VjCdy5RaVuCZ6ajTif7Kq+BEOE8TumYx4ly
zdhnh/9ICohqfVvEMh347wI36D7HuezHB773hOsHdqTy9T+0Qu0Vu+wud45MUy1f
vRF11OkJFtKL0bh4yMSGVY1xte1Mt/qC6rd43TDtAW3ekw1o/exh764kp7XXQsmP
wwe4Y040PZafcygJlEW9bBtjjxKnzDTvqeb5dMi6d7a0GENocmlzIDxvaWRldkBz
dW5vcy5pbmZvPokBaAQQAQgAUgYLCQcIAwIEFQgKAgMWAgECGwMCHgEYGGhrcHM6
Ly9rZXlzLm9wZW5wZ3Aub3JnFiEEBiQLMsgZZHLsLEmLGHUefL3klUAFAmDUJfsF
CQPCwJoACgkQGHUefL3klUB74wf8DSvT36bYZp7oqZ+35HNhTekJ2dbTzUhauF0S
+Z9R1AGnNnINgua75CyQGdNCIgcZxo4qG9sePl7SllQ9i0qhmiw0mzmvky8bAZQV
V/2Coc1C/81b+PI19VczYrbZC20jApsnbAIkKZgSh9XQoiLd3meY7G2lX2k6CXYL
xSeBEh+N3BU8vLxExm82U71Qzm43u0kA1TlbTSqpBvg/tfAzTCsYQLSlB6b4ZL2W
D6U7b7ZYF5oZNonVNWSHxpjUN3Evkta9xWS2+cgYQdlP1/ku5w5ZWwzmYG7awh0J
/YuSNIp6Ks6D/PSBduu6XbH+FJHaXmq+ZCKpNBh5EKH+GhOfq7QfQ2hyaXMgPHBv
cnRtYXN0ZXJAYnNkZm9yZ2UuY29tPokBaAQQAQgAUgYLCQcIAwIEFQgKAgMWAgEC
GwMCHgEYGGhrcHM6Ly9rZXlzLm9wZW5wZ3Aub3JnFiEEBiQLMsgZZHLsLEmLGHUe
fL3klUAFAmDUJfwFCQPCwJoACgkQGHUefL3klUC3GggAo4Y+hslaoV7Namp7qWYZ
Vei4ZwPfsYW7/HtmFORSGV8C8xR+LSkwzN1Hc7Qxvwv+DXuk7Hzd1Ag/xe8XhbNG
/NMrXENY/8ym9TRbxtrBIhQyhkyShSUT+N+g16GRNZKuNL2MOIHc/RCS/YyyaTtu
TzIxFbP7Gb2LO1LiiZsFVOGirHfxyiww7CAm3HXY2K4smOiKs6swZMpStVy3dd6A
BcB1LPGs3ywDglFfKCRbVmjsPgsi61r4kUBVO6ML7lAmPDXLXOa+7iAtBN479QxC
MVeH3Y3SMrvu61Vyf1xL79rIznU3u8C34zfxqsoIV0zCZe2YDLbFfLhZYqatYYEo
e7QjImNocmlzLmgiIDxjaHJpcy5oQHVsdGltYXRlZG5zLm5ldD6JAWgEEAEIAFIG
CwkHCAMCBBUICgIDFgIBAhsDAh4BGBhoa3BzOi8va2V5cy5vcGVucGdwLm9yZxYh
BAYkCzLIGWRy7CxJixh1Hny95JVABQJg1CX8BQkDwsCaAAoJEBh1Hny95JVAkUEH
/jkzYrRh7muqoebwEgVeULzPbAs/nYJm9SMME2ypB2FS8kusO7lE+33UJO7PhHkJ
0nJ+tPfP8UV+fCzVjKjabzpvUGuiMWKRZEK9xNoxwi/epOrRw87msHA2LPqEob+F
sVh09Nc58s75koUgSYp5h0FjsLK0+fwsQ6PtTfpY5W6JJVJRQnMwGKk5czrukBSM
79kJvphgul2xuzqo5K7rM98dL75AwCJmJZnbyXpUJIhtY/G01nURupBiQGgNixYs
Zeo6OR669TFrMRWxueXtlHD0WaX7JNSlR5uyzpVaDCH0Kxa6ozmZtD+a6dAXg630
zbLGHg51JIm38Uvi1i47Jaa0KCJILlIuIENvbW11bmljYXRpb25zIiA8ZG5zQGRu
c3dhdGNoLmNvbT6JAWgEEAEIAFIGCwkHCAMCBBUICgIDFgIBAhsDAh4BGBhoa3Bz
Oi8va2V5cy5vcGVucGdwLm9yZxYhBAYkCzLIGWRy7CxJixh1Hny95JVABQJg1CX8
BQkDwsCaAAoJEBh1Hny95JVAABoH/iOWA+9BKxLIAIFgW2nxTFDrGvbxXL/mVSFt
SOInKX8UqqfLCcikfpWLsj2D7mg5rKFMCu+31UYYlnrXl4YY1qruq0vh41L72qNy
yHYol+xW4BSbZXf2q2ph7+lnPsFoodw7acVun5F8M8NH0roo5AOSbgRlK69ZFIcq
fDEJdtk4oul7pqGArdeTCCdrSaeR3zrRN8P0PDOkGKSdlpeOE6XHnbbmAPZIhr/9
KsSpX1BGyipda3k5kOB4TsGVo+cRJMkK+GMpsZ+lJ7ZzRbjHbC+b52TiAIjMtXCK
3A3LrDUeMoJwvRKoO1tzquF6HqHJSg0ArZOvAB3BHlwUyUtA/o25AQ0EYNPMYQEI
ANFpucNRdYEOubTNluoK97N9JmDb0WRXPPow+3XfBom6ZBSrWqNBgqDbjxSsLB00
QXbA8EB5W/Oolp/0epwEtgNAxyKVPowE/un+rY1PqvGjeAR4gBhY9Za1Lg1Q3vnR
/WzsY7RIQCqhWUbfdGn1u6r/EgTBVrwUp4U/3ggfSz/PcUt4pUhlgxfYvjSjOgEZ
wbqaQIwWud11FKMARNAUJzvJL/fDGeKLMvgRUwynIDGzCq7e67hhEEo5jwkZ0gEl
8RxXHKFuYkbb/q7rpdifXYYT6QCFlEZhiRbtH5Us7kgKuRD2XUFEQnN4U/rxuydH
4XOP6iOhiZfYnK/y9HBeRCMAEQEAAYkBPAQYAQgAJgIbDBYhBAYkCzLIGWRy7CxJ
ixh1Hny95JVABQJg1CYkBQkDwsDDAAoJEBh1Hny95JVApBsH/iEg2ANRkHByfXB+
sH3PMf2Jsg5NSuj8OiNeKKGGIKCJkSAPjtv5rvKLNcvIcTR5Vnhr0e6AteFcK2te
iFWDmj0QuFoQNvIOHQ3nHBPSpai2Ubq12nvYfg4bYK28AMi4xPMssgQ8awFgAI2V
k9okq5XwC0Cc1MGhupEWYYSaFLIDQvFvRRSw1Lyc/W3SKa4d2dgesIPnB/rdv0Zq
u8ftsSmurKxA2hQeNIcn06Ew7AbWUIjFX/bDXJlg/3Sj/spU2ur23TmaADBKhT5P
DvfdaFTkk0SBfpN1j2S0DNXBHSrWvRp15zZmU4hwELiUY/H2/j/XpOGV3Q0i2iob
1hJ30C8=
=aMQi
-----END PGP PUBLIC KEY BLOCK-----
--=_561093c953b62cf25d5e022249f2d35a--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?500d4bb5b672b8c3fac18bf76f781230>