Date: Sun, 19 Apr 2020 10:56:47 -0600 From: Ian Lepore <ian@freebsd.org> To: Andrey Smagin <samspeed@mail.ru>, freebsd-arm@freebsd.org Subject: Re: Multithreaded latency too random. Message-ID: <ddc304376858b5a043813bdc7f5724b2516bce81.camel@freebsd.org> In-Reply-To: <1587258233.899122295@f226.i.mail.ru> References: <1587258233.899122295@f226.i.mail.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 2020-04-19 at 04:03 +0300, Andrey Smagin via freebsd-arm wrote:
> I wrote simple multithreaded example :
>
> main.cpp:
> #include <thread>
> #include <unistd.h>
> #include <time.h>
> int dTimeMax = 0;
> int dTimeMin = 0x7FFFFFFF;
> int Nanosec;
> bool ThreadExit = false;
> void Thread()
> {
> timespec T1,T2;
> for( ; !ThreadExit ; )
> {
> clock_gettime( CLOCK_REALTIME_PRECISE, &T1 );
> clock_gettime( CLOCK_REALTIME_PRECISE, &T2 );
> Nanosec = ( T2.tv_sec - T1.tv_sec )*1000000000 + T2.tv_nsec -
> T1.tv_nsec ;
> if ( Nanosec > dTimeMax ) dTimeMax = Nanosec;
> if ( Nanosec < dTimeMin ) dTimeMin = Nanosec;
> }
> }
> int main( int argc, char *argv[] )
> {
> std::thread Thr( Thread );
> for( int N=0; N<100000; ++N )
> printf("%d \t%d \t%d\n", dTimeMax, dTimeMin, Nanosec );
> ThreadExit=true;
> Thr.join();
> }
>
> On my PC
> dTimeMax — dTimeMin < 8000 in console over ssh.
> dTimeMax — dTimeMin < 5000 in tmpfs file output.
>
> On my Orange-Pi Zero board it produce very random values:
> dTimeMax — dTimeMin > 2 000 000 in console over ssh .
> dTimeMax — dTimeMin > 15 000 in tmpfs file output.
>
> What can help disable interrupt of thread ?
>
> Set kern.hz=250000 not change result.
>
> FreeBSD orangepi-zero 13.0-CURRENT FreeBSD 13.0-CURRENT #0 r359929:
> Thu Apr 16 06:10:42 MSK 2020 /usr/src/arm.armv7/sys/GENERIC-NODEBUG
> arm
>
> to compile: c++ main.cpp -lthr
>
>
>
The only thing I find surprising about that is the 2ms of variability
in the arm console over ssh case. The 15us in the tmpfs case is
actually better than I would have guessed.
I wonder what it is about ssh over console? Is it the network
activity? It might be interesting to run the tmpfs test while having
iperf or something similiar running in another window to load up the
network.
I wonder if dtrace might be able to answer any questions here? I'm not
very good with dtrace, but I suspect we have some people reading here
who might suggest some of the canned dtrace scripts that could help
figure out what's causing the high variability in timing in the ssh
case.
kern.hz doesn't really control anything anymore like it did in the old
days. It has some indirect effects on performance, but nothing like it
did in the days before eventtimers and the tickless kernel changes.
-- Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ddc304376858b5a043813bdc7f5724b2516bce81.camel>