Date: Wed, 23 Oct 2024 15:14:08 -0400 From: Cheng Cui <cc@freebsd.org> To: void <void@f-m.fm> Cc: freebsd-net@freebsd.org Subject: Re: Performance test for CUBIC in stable/14 Message-ID: <CAGaXuiLuuXW_gFMq=--1Z2rKZ4ZZUiV52BnW10FxDYZc6vBZ-Q@mail.gmail.com> In-Reply-To: <ZxkQiHWlDz28az-N@vm2> References: <ZxEqfISBJIW75p_P@vm2> <CAGaXuiJW5QW1eFqjXtSO%2BojOX5G12B56%2BnzE9zaVR7nB33VjWA@mail.gmail.com> <ZxJe8e8sRU9NCHv4@vm2> <CAGaXuiKD-b4PGrqfxy9zk-BRxU==HMc9KshqyJGzH8saeOLf1A@mail.gmail.com> <ZxaccxFblDt0UQWR@int21h> <CAGaXui%2BQ7wCM1dAKis%2BvNaNJ5uODeiC765hoXWT4OBtT7npprw@mail.gmail.com> <ZxfFRg3tYtdQt0hM@vm2> <CAGaXuiJvC2i5yxEaaDmHMoadPzkk3oqQOzg1yiqBuhTR%2B=R9Sg@mail.gmail.com> <ZxgHOlRaCR6Joqdv@vm2> <CAGaXui%2BEYmRhOdwOqFRbVsboCkrrWmnHnWRSqsSAgzbn5ug6bg@mail.gmail.com> <ZxkQiHWlDz28az-N@vm2>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000de1bc8062529b2fb Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I see. The result of `newreno` vs. `cubic` shows non-constant/infrequent packet retransmission. So TCP congestion control has little impact on improving th= e performance. The performance bottleneck may come from somewhere else. For example, the sender CPU shows 97.7% utilization. Would there be any way to reduce CPU usage? cc On Wed, Oct 23, 2024 at 11:04=E2=80=AFAM void <void@f-m.fm> wrote: > On Wed, Oct 23, 2024 at 08:28:01AM -0400, Cheng Cui wrote: > >The latency does not sound a problem to me. What is the performance of > >TCP congestion control algorithm `newreno`? > > > >In case you need to load `newreno` first. > > > >cc@n1:~ % sudo kldload newreno > > > >cc@n1:~ % sudo sysctl net.inet.tcp.cc.algorithm=3Dnewreno > > > >net.inet.tcp.cc.algorithm: cubic -> newreno > > > >cc@n1:~ % > > > >And let me know the result of `newreno` vs. `cubic`, for example: > >iperf3 -B ${src} --cport ${tcp_port} -c ${dst} -l 1M -t 20 -i 2 -VC > newreno > > speedtests@vm4-fbsd14s:~ % doas kldload newreno > speedtests@vm4-fbsd14s:~ % doas sysctl net.inet.tcp.cc.algorithm=3Dnewren= o > net.inet.tcp.cc.algorithm: cubic -> newreno > > speedtests@vm4-fbsd14s:~ % iperf3 -B 192.168.1.13 --cport 5201 -c > 192.168.1.232 > -l 1M -t 20 -i 2 -VC newreno > iperf 3.17.1 > FreeBSD vm4-fbsd14s.home.arpa 14.2-PRERELEASE FreeBSD 14.2-PRERELEASE #0 > stable/14-n269252-e18ba5c5555a-dirty: Mon Oct 21 18:09:22 BST 2024 > root@vm4-fbsd14s.home.arpa:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 > Control connection MSS 1460 > Time: Wed, 23 Oct 2024 14:41:11 UTC > Connecting to host 192.168.1.232, port 5201 > Cookie: tvrlkd2axzx24uui7gglzk4ni66ib7qy4kxa > TCP MSS: 1460 (default) > [ 5] local 192.168.1.13 port 5201 connected to 192.168.1.232 port 5201 > Starting Test: protocol: TCP, 1 streams, 1048576 byte blocks, omitting 0 > seconds, > 20 second test, tos 0 > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-2.01 sec 137 MBytes 572 Mbits/sec 0 629 KBytes > > [ 5] 2.01-4.13 sec 159 MBytes 628 Mbits/sec 0 928 KBytes > > [ 5] 4.13-6.12 sec 192 MBytes 809 Mbits/sec 0 1.16 MBytes > > [ 5] 6.12-8.08 sec 153 MBytes 656 Mbits/sec 0 1.33 MBytes > > [ 5] 8.08-10.08 sec 176 MBytes 737 Mbits/sec 0 1.51 MBytes > > [ 5] 10.08-12.13 sec 211 MBytes 864 Mbits/sec 0 1.69 MBytes > > [ 5] 12.13-14.04 sec 138 MBytes 606 Mbits/sec 73 1.01 MBytes > > [ 5] 14.04-16.02 sec 155 MBytes 657 Mbits/sec 0 1.21 MBytes > > [ 5] 16.02-18.10 sec 168 MBytes 678 Mbits/sec 0 1.39 MBytes > > [ 5] 18.10-20.13 sec 188 MBytes 777 Mbits/sec 0 1.56 MBytes > > - - - - - - - - - - - - - - - - - - - - - - - - - > Test Complete. Summary Results: > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-20.13 sec 1.64 GBytes 699 Mbits/sec 73 sender > [ 5] 0.00-20.14 sec 1.64 GBytes 698 Mbits/sec receiver > CPU Utilization: local/sender 97.7% (0.0%u/97.7%s), remote/receiver 19.5% > (1.6%u/17.9%s) > snd_tcp_congestion newreno > rcv_tcp_congestion newreno > > iperf Done. > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > speedtests@vm4-fbsd14s:~ % doas sysctl net.inet.tcp.cc.algorithm=3Dcubic > net.inet.tcp.cc.algorithm: newreno -> cubic > > speedtests@vm4-fbsd14s:~ % iperf3 -B 192.168.1.13 --cport 5201 -c > 192.168.1.232 > -l 1M -t 20 -i 2 -VC cubic > iperf 3.17.1 > FreeBSD vm4-fbsd14s.home.arpa 14.2-PRERELEASE FreeBSD 14.2-PRERELEASE #0 > stable/14-n269252-e18ba5c5555a-dirty: Mon Oct 21 18:09:22 BST 2024 > root@vm4-fbsd14s.home.arpa:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 > Control connection MSS 1460 > Time: Wed, 23 Oct 2024 14:51:30 UTC > Connecting to host 192.168.1.232, port 5201 > Cookie: wp5nkovyy5pwzqos4lsdlqv4loccl6iu5kdv > TCP MSS: 1460 (default) > [ 5] local 192.168.1.13 port 5201 connected to 192.168.1.232 port 5201 > Starting Test: protocol: TCP, 1 streams, 1048576 byte blocks, > omitting 0 seconds, 20 second test, tos 0 > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-2.03 sec 184 MBytes 762 Mbits/sec 0 752 KBytes > > [ 5] 2.03-4.07 sec 198 MBytes 811 Mbits/sec 0 1.05 MBytes > > [ 5] 4.07-6.13 sec 193 MBytes 787 Mbits/sec 0 1.28 MBytes > > [ 5] 6.13-8.08 sec 203 MBytes 874 Mbits/sec 0 1.48 MBytes > > [ 5] 8.08-10.13 sec 192 MBytes 786 Mbits/sec 0 1.65 MBytes > > [ 5] 10.13-12.13 sec 156 MBytes 653 Mbits/sec 44 1.40 MBytes > > [ 5] 12.13-14.13 sec 167 MBytes 703 Mbits/sec 16 1.04 MBytes > > [ 5] 14.13-16.07 sec 167 MBytes 721 Mbits/sec 0 1.25 MBytes > > [ 5] 16.07-18.02 sec 114 MBytes 490 Mbits/sec 0 1.37 MBytes > > [ 5] 18.02-20.04 sec 173 MBytes 719 Mbits/sec 0 1.53 MBytes > > - - - - - - - - - - - - - - - - - - - - - - - - - > Test Complete. Summary Results: > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-20.04 sec 1.71 GBytes 731 Mbits/sec 60 > sender > [ 5] 0.00-20.05 sec 1.71 GBytes 730 Mbits/sec > receiver > CPU Utilization: local/sender 97.6% (0.0%u/97.6%s), remote/receiver > 20.5% (1.8%u/18.6%s) > snd_tcp_congestion cubic > rcv_tcp_congestion cubic > > iperf Done. > speedtests@vm4-fbsd14s:~ % > > -- > > --=20 Best Regards, Cheng Cui --000000000000de1bc8062529b2fb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>I see. The result of `newreno` vs. `cubic` shows non-= constant/infrequent packet</div><div>retransmission. So TCP congestion cont= rol has little impact on improving=C2=A0the</div><div>performance.</div><di= v><br></div><div>The performance bottleneck=C2=A0may come=C2=A0from somewhe= re else. For=C2=A0example, the</div><div>sender CPU shows 97.7% utilization= . Would there be any way to reduce CPU</div><div>usage?<br></div><div><br><= /div><div>cc</div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"= gmail_attr">On Wed, Oct 23, 2024 at 11:04=E2=80=AFAM void <<a href=3D"ma= ilto:void@f-m.fm">void@f-m.fm</a>> wrote:<br></div><blockquote class=3D"= gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border= -left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">On W= ed, Oct 23, 2024 at 08:28:01AM -0400, Cheng Cui wrote:<br> >The latency does not sound a problem to me. What is the performance of<= br> >TCP congestion control algorithm `newreno`?<br> ><br> >In case you need to load `newreno` first.<br> ><br> >cc@n1:~ % sudo kldload newreno<br> ><br> >cc@n1:~ % sudo sysctl net.inet.tcp.cc.algorithm=3Dnewreno<br> ><br> >net.inet.tcp.cc.algorithm: cubic -> newreno<br> ><br> >cc@n1:~ %<br> ><br> >And let me know the result of `newreno` vs. `cubic`, for example:<br> >iperf3 -B ${src} --cport ${tcp_port} -c ${dst} -l 1M -t 20 -i 2 -VC new= reno<br> <br> speedtests@vm4-fbsd14s:~ % doas kldload newreno<br> speedtests@vm4-fbsd14s:~ % doas sysctl net.inet.tcp.cc.algorithm=3Dnewreno<= br> net.inet.tcp.cc.algorithm: cubic -> newreno<br> <br> speedtests@vm4-fbsd14s:~ % iperf3 -B 192.168.1.13 --cport 5201 -c 192.168.1= .232<br> -l 1M -t 20 -i 2 -VC newreno<br> iperf 3.17.1<br> FreeBSD vm4-fbsd14s.home.arpa 14.2-PRERELEASE FreeBSD 14.2-PRERELEASE #0<br= > stable/14-n269252-e18ba5c5555a-dirty: Mon Oct 21 18:09:22 BST 2024<br> root@vm4-fbsd14s.home.arpa:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64<b= r> Control connection MSS 1460<br> Time: Wed, 23 Oct 2024 14:41:11 UTC<br> Connecting to host 192.168.1.232, port 5201<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0Cookie: tvrlkd2axzx24uui7gglzk4ni66ib7qy4kxa<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0TCP MSS: 1460 (default)<br> [=C2=A0 5] local 192.168.1.13 port 5201 connected to 192.168.1.232 port 520= 1<br> Starting Test: protocol: TCP, 1 streams, 1048576 byte blocks, omitting 0 se= conds, <br> 20 second test, tos 0<br> [ ID] Interval=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Transfer=C2=A0 =C2= =A0 =C2=A0Bitrate=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Retr=C2=A0 Cwnd<br> [=C2=A0 5]=C2=A0 =C2=A00.00-2.01=C2=A0 =C2=A0sec=C2=A0 =C2=A0137 MBytes=C2= =A0 =C2=A0572 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A0 629 KBytes=C2=A0 =C2=A0= =C2=A0 =C2=A0<br> [=C2=A0 5]=C2=A0 =C2=A02.01-4.13=C2=A0 =C2=A0sec=C2=A0 =C2=A0159 MBytes=C2= =A0 =C2=A0628 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A0 928 KBytes=C2=A0 =C2=A0= =C2=A0 =C2=A0<br> [=C2=A0 5]=C2=A0 =C2=A04.13-6.12=C2=A0 =C2=A0sec=C2=A0 =C2=A0192 MBytes=C2= =A0 =C2=A0809 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A01.16 MBytes=C2=A0 =C2=A0= =C2=A0 =C2=A0<br> [=C2=A0 5]=C2=A0 =C2=A06.12-8.08=C2=A0 =C2=A0sec=C2=A0 =C2=A0153 MBytes=C2= =A0 =C2=A0656 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A01.33 MBytes=C2=A0 =C2=A0= =C2=A0 =C2=A0<br> [=C2=A0 5]=C2=A0 =C2=A08.08-10.08=C2=A0 sec=C2=A0 =C2=A0176 MBytes=C2=A0 = =C2=A0737 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A01.51 MBytes=C2=A0 =C2=A0 =C2= =A0 =C2=A0<br> [=C2=A0 5]=C2=A0 10.08-12.13=C2=A0 sec=C2=A0 =C2=A0211 MBytes=C2=A0 =C2=A08= 64 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A01.69 MBytes=C2=A0 =C2=A0 =C2=A0 =C2= =A0<br> [=C2=A0 5]=C2=A0 12.13-14.04=C2=A0 sec=C2=A0 =C2=A0138 MBytes=C2=A0 =C2=A06= 06 Mbits/sec=C2=A0 =C2=A073=C2=A0 =C2=A01.01 MBytes=C2=A0 =C2=A0 =C2=A0 =C2= =A0<br> [=C2=A0 5]=C2=A0 14.04-16.02=C2=A0 sec=C2=A0 =C2=A0155 MBytes=C2=A0 =C2=A06= 57 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A01.21 MBytes=C2=A0 =C2=A0 =C2=A0 =C2= =A0<br> [=C2=A0 5]=C2=A0 16.02-18.10=C2=A0 sec=C2=A0 =C2=A0168 MBytes=C2=A0 =C2=A06= 78 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A01.39 MBytes=C2=A0 =C2=A0 =C2=A0 =C2= =A0<br> [=C2=A0 5]=C2=A0 18.10-20.13=C2=A0 sec=C2=A0 =C2=A0188 MBytes=C2=A0 =C2=A07= 77 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A01.56 MBytes=C2=A0 =C2=A0 =C2=A0 =C2= =A0<br> - - - - - - - - - - - - - - - - - - - - - - - - -<br> Test Complete. Summary Results:<br> [ ID] Interval=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Transfer=C2=A0 =C2= =A0 =C2=A0Bitrate=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Retr<br> [=C2=A0 5]=C2=A0 =C2=A00.00-20.13=C2=A0 sec=C2=A0 1.64 GBytes=C2=A0 =C2=A06= 99 Mbits/sec=C2=A0 =C2=A073 sender<br> [=C2=A0 5]=C2=A0 =C2=A00.00-20.14=C2=A0 sec=C2=A0 1.64 GBytes=C2=A0 =C2=A06= 98 Mbits/sec receiver<br> CPU Utilization: local/sender 97.7% (0.0%u/97.7%s), remote/receiver 19.5% (= 1.6%u/17.9%s)<br> snd_tcp_congestion newreno<br> rcv_tcp_congestion newreno<br> <br> iperf Done.<br> <br> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br> <br> speedtests@vm4-fbsd14s:~ % doas sysctl net.inet.tcp.cc.algorithm=3Dcubic<br= > net.inet.tcp.cc.algorithm: newreno -> cubic<br> <br> speedtests@vm4-fbsd14s:~ % iperf3 -B 192.168.1.13 --cport 5201 -c 192.168.1= .232<br> -l 1M -t 20 -i 2 -VC cubic<br> iperf 3.17.1<br> FreeBSD vm4-fbsd14s.home.arpa 14.2-PRERELEASE FreeBSD 14.2-PRERELEASE #0<br= > stable/14-n269252-e18ba5c5555a-dirty: Mon Oct 21 18:09:22 BST 2024<br> root@vm4-fbsd14s.home.arpa:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64<b= r> Control connection MSS 1460<br> Time: Wed, 23 Oct 2024 14:51:30 UTC<br> Connecting to host 192.168.1.232, port 5201<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0Cookie: wp5nkovyy5pwzqos4lsdlqv4loccl6iu5kdv<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0TCP MSS: 1460 (default)<br> [=C2=A0 5] local 192.168.1.13 port 5201 connected to 192.168.1.232 port 520= 1<br> Starting Test: protocol: TCP, 1 streams, 1048576 byte blocks,<br> omitting 0 seconds, 20 second test, tos 0<br> [ ID] Interval=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Transfer=C2=A0 =C2= =A0 =C2=A0Bitrate=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Retr=C2=A0 Cwnd<br> [=C2=A0 5]=C2=A0 =C2=A00.00-2.03=C2=A0 =C2=A0sec=C2=A0 =C2=A0184 MBytes=C2= =A0 =C2=A0762 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A0 752 KBytes=C2=A0 =C2=A0= =C2=A0 =C2=A0<br> [=C2=A0 5]=C2=A0 =C2=A02.03-4.07=C2=A0 =C2=A0sec=C2=A0 =C2=A0198 MBytes=C2= =A0 =C2=A0811 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A01.05 MBytes=C2=A0 =C2=A0= =C2=A0 =C2=A0<br> [=C2=A0 5]=C2=A0 =C2=A04.07-6.13=C2=A0 =C2=A0sec=C2=A0 =C2=A0193 MBytes=C2= =A0 =C2=A0787 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A01.28 MBytes=C2=A0 =C2=A0= =C2=A0 =C2=A0<br> [=C2=A0 5]=C2=A0 =C2=A06.13-8.08=C2=A0 =C2=A0sec=C2=A0 =C2=A0203 MBytes=C2= =A0 =C2=A0874 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A01.48 MBytes=C2=A0 =C2=A0= =C2=A0 =C2=A0<br> [=C2=A0 5]=C2=A0 =C2=A08.08-10.13=C2=A0 sec=C2=A0 =C2=A0192 MBytes=C2=A0 = =C2=A0786 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A01.65 MBytes=C2=A0 =C2=A0 =C2= =A0 =C2=A0<br> [=C2=A0 5]=C2=A0 10.13-12.13=C2=A0 sec=C2=A0 =C2=A0156 MBytes=C2=A0 =C2=A06= 53 Mbits/sec=C2=A0 =C2=A044=C2=A0 =C2=A01.40 MBytes=C2=A0 =C2=A0 =C2=A0 =C2= =A0<br> [=C2=A0 5]=C2=A0 12.13-14.13=C2=A0 sec=C2=A0 =C2=A0167 MBytes=C2=A0 =C2=A07= 03 Mbits/sec=C2=A0 =C2=A016=C2=A0 =C2=A01.04 MBytes=C2=A0 =C2=A0 =C2=A0 =C2= =A0<br> [=C2=A0 5]=C2=A0 14.13-16.07=C2=A0 sec=C2=A0 =C2=A0167 MBytes=C2=A0 =C2=A07= 21 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A01.25 MBytes=C2=A0 =C2=A0 =C2=A0 =C2= =A0<br> [=C2=A0 5]=C2=A0 16.07-18.02=C2=A0 sec=C2=A0 =C2=A0114 MBytes=C2=A0 =C2=A04= 90 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A01.37 MBytes=C2=A0 =C2=A0 =C2=A0 =C2= =A0<br> [=C2=A0 5]=C2=A0 18.02-20.04=C2=A0 sec=C2=A0 =C2=A0173 MBytes=C2=A0 =C2=A07= 19 Mbits/sec=C2=A0 =C2=A0 0=C2=A0 =C2=A01.53 MBytes=C2=A0 =C2=A0 =C2=A0 =C2= =A0<br> - - - - - - - - - - - - - - - - - - - - - - - - -<br> Test Complete. Summary Results:<br> [ ID] Interval=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Transfer=C2=A0 =C2= =A0 =C2=A0Bitrate=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Retr<br> [=C2=A0 5]=C2=A0 =C2=A00.00-20.04=C2=A0 sec=C2=A0 1.71 GBytes=C2=A0 =C2=A07= 31 Mbits/sec=C2=A0 =C2=A060<br> sender<br> [=C2=A0 5]=C2=A0 =C2=A00.00-20.05=C2=A0 sec=C2=A0 1.71 GBytes=C2=A0 =C2=A07= 30 Mbits/sec<br> receiver<br> CPU Utilization: local/sender 97.6% (0.0%u/97.6%s), remote/receiver<br> 20.5% (1.8%u/18.6%s)<br> snd_tcp_congestion cubic<br> rcv_tcp_congestion cubic<br> <br> iperf Done.<br> speedtests@vm4-fbsd14s:~ % <br> <br> -- <br> <br> </blockquote></div><br clear=3D"all"><div><br></div><span class=3D"gmail_si= gnature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><d= iv dir=3D"ltr"><div></div>Best Regards,<div>Cheng Cui</div></div></div></di= v> --000000000000de1bc8062529b2fb--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAGaXuiLuuXW_gFMq=--1Z2rKZ4ZZUiV52BnW10FxDYZc6vBZ-Q>