From owner-freebsd-threads@freebsd.org Fri Jun 3 04:16:48 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2DD8B6874D for ; Fri, 3 Jun 2016 04:16:48 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: from mail-oi0-x22b.google.com (mail-oi0-x22b.google.com [IPv6:2607:f8b0:4003:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 82E891293 for ; Fri, 3 Jun 2016 04:16:48 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: by mail-oi0-x22b.google.com with SMTP id j1so109318331oih.3 for ; Thu, 02 Jun 2016 21:16:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sippysoft-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:date:message-id:subject:from:to; bh=UuUa4dXL1nc6kcqGLYHhwZ8+IbnuB6uwDNezDYUB6qM=; b=PALyFcIuB5PPzuTvm9CxO1PXGTYU+vNmxXwUppovywJPTdjlZMmzmM/yhVyi3+TRQE hGdhzBU7O45vpIBMun7yM6/TfPlBItIXD8p+qscURFANU5wRAcESAkeagjNumkcpH7nG TRG4kK9hsz5qiv5jEs3C39v7O4/hp9KRry9gajx6fuu9fjY1ql3dBEuPcuQ4vPV2t/V1 z7t+vYZeeyAAOHbHVMYa70zy3UKWdoWzsmlqs75a8zhIvBvh8RXGj4QX8XGD/FvGS6zb pCEOGzbxiLFw1F/UDUjqgfqrP9fBO09XGFIKYPqqhVPpubYoEkLp28nefXrCCtEXiTWi gmmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:date:message-id:subject:from :to; bh=UuUa4dXL1nc6kcqGLYHhwZ8+IbnuB6uwDNezDYUB6qM=; b=GcGcyGApNXlHif3wqvs+cU4qa2N+ar8wgQ51Ueu2Hpe+Lzuy7r7GM1PWQetPpP1kS8 TzOT2lFyGOEZbdkDx5xYzXGCs2D/N+z/DgQ3fEU+CgO5CX3Yz5G0U9D+KJhhPuRhneZz AUALfh9HtY2RbRKStpu+bX1KagKoNMU6iyzTEPKzXJZk3qxpOaR4AvYVptYaEJ1eKiwB fG/QssVv0QopNivZlBYHTVDKx2FJBxxBpzS5yQfscLra+RmjYZX38o3bAUYtqM7zIcLG DWt4+APAeBd/aKilEOu5ZKt0Qw1cS9V96laXNFgPFqt1vO7WDcV2orEsUDR2rkYpFnB0 gQAg== X-Gm-Message-State: ALyK8tIrsQbomwdn3RgHDUl/pMZEtMoZuXlAUWajsyfoUsSpvGBNa7EKk5TlPxNZhvk3/JFg2TFD5NctDrQcSAJA MIME-Version: 1.0 X-Received: by 10.202.241.67 with SMTP id p64mr652858oih.197.1464927407613; Thu, 02 Jun 2016 21:16:47 -0700 (PDT) Sender: sobomax@sippysoft.com Received: by 10.157.56.70 with HTTP; Thu, 2 Jun 2016 21:16:47 -0700 (PDT) Date: Thu, 2 Jun 2016 21:16:47 -0700 X-Google-Sender-Auth: _pVd8Ya48XkdEhMcg4Hhyl0Z93I Message-ID: Subject: CLOCK_MONOTONIC / CLOCK_UPTIME is not really monotonic between threads From: Maxim Sobolev To: FreeBSD Current , freebsd-threads@freebsd.org, freebsd-questions@freebsd.org, freebsd-arch@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Jun 2016 04:16:48 -0000 Hi there, we have an application here which is trying to measure UDP command/response round-trip-time. It runs two posix threads (more actually, but that's probably irrelevant), one (let's call it A) that does high-level logic and the second one (B) that does network packet I/O. The sending side is done by first thread A forming the request, then calling the function clock_gettime(CLOCK_MONOTONIC) and passing the packet into the thread B. Obtained timestamp is stored with some logical transaction ID allowing us to pull that stored value later on when response arrives. Then we have a separate process that receives those requests, processing them and sending back some form of response. Upon receiving a response from the network, the network I/O thread (B) timestamps it by running clock_gettime(CLOCK_MONOTONIC) and passes the packet data along with that value via queue to the thread A for processing. So if we put things into timeline, what our app does would probably look something like the following: 1. Thread A generates request. 2. A calls clock_gettime(CLOCK_MONOTONIC), storing value as t1 internally 3. A passes packet to thread B 4. B sends out packet via sendto() to server process running on the same box (fully separate, not a thread) [some microseconds later] 5. B receives response from server with recvfrom() 6. B instantly calls clock_gettime(CLOCK_MONOTONIC), assigns returned value to t2 7. B passes packet data along with t2 to the A via queue 8. A picks up packet, parses it and retrieves corresponding t1 stored at step 2. 9. A calculates RTT by doing t2 - t1 assuming it's going to be positive... As you might have guessed if you are still reading, from time to time t2 - t1 comes out slightly negative! Provided it's not some obscure bug in our app, there is no way this could happen if clock_gettime(CLOCK_MONOTONIC) would work as advertised. Event (2) could not possibly happen earlier than (6), which is guaranteed by the fact that the request needs to be processed by the external entity first in order for the response to be seen by our app at all. I've added some logs and it seems to be confirming that the server only sees a single request, there is no chance for the client to receive some other packet and confuse it. I've also confirmed with tcpdump, which shows reasonable time delay between request and reply of few hundreds microseconds. I've checked all logic and I could not find any mistakes on my end here, so I added some logging for such events. The distribution appears to be centered around 0.00006s, but there are some events that go as far up as 0.012s. I've also tried using CLOCK_UPTIME_PRECISE instead, but it makes no difference whatsoever. My questions therefore are: 1. Is it intended/expected behavior of the said API? 2. Has anyone else bumped into this? 3. I know we are doing some clever optimizations using TSC to speed those APIs up, could be it related to that? 4. If the answer for (3) is yes, then what is the method to disable using TSC and use slower but possibly more reliable method? 5. Is there any way/plan to fix this long-term? 6. If the behavior is intended/expected, what is the maximum discrepancy we can expect from this effect? In general some time ago we've spend quite lot of time switching our app from using REALTIME into MONOTIME in the hope to eliminate any wizardry needed to deal with realtime possibly jumping back and forth due to NTP, leap seconds etc, it's shame that this is not working either. Apart from measuring command processing delay, that app does lot of high-volume voip call billing, so even such small discrepancy can easily build up into a bigger problem, not to mention the fact that every time we deal with duration we now need to have some check in place to make sure we don't have some negative values popping our of nowhere. Any hints, suggestions, pointers are appreciated. I can also give more debug information as needed. Thanks! -Max From owner-freebsd-threads@freebsd.org Fri Jun 3 05:05:29 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 991DBB68FDA; Fri, 3 Jun 2016 05:05:29 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-io0-x231.google.com (mail-io0-x231.google.com [IPv6:2607:f8b0:4001:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 653831B40; Fri, 3 Jun 2016 05:05:29 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-io0-x231.google.com with SMTP id o189so54931278ioe.2; Thu, 02 Jun 2016 22:05:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=HB2MCHDu3HgA9EshZyPtgWRG8azVzZUZFLIwjsNXc4A=; b=qvqAYuCxTCebd03wL9RXt5fe999NOWxcnFvdYKRzRmVqkS4CGskzREs/3cfjwk6ru0 ZDSEzO40J5bEzWG8vkvqT1t1XOIDvXOzvvouRCAPbBWPjLB0mCXsAIPFaZM2ZYMZU8ko 5qUTI40ph0s31Zh6pnVH3pROoVuXinWjckIXazrKqdClFvvVZhehbCbkfDVKsJOyeEOq Puislektl9aLQ1Awp4PmRRCu1CGd7tY0n6iuuFnKEA84L2ArB6LDJYSzMHl0Q8fDBlLE 9KqHo0H6YAiQ5nAOGE0/x+WCRWVyjpNfwpUPDdBWO4/8SzVxi+gxOsG/yGYugO+DPNdc 0IMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=HB2MCHDu3HgA9EshZyPtgWRG8azVzZUZFLIwjsNXc4A=; b=ltZi74TQWgTRBiw61zM5zUv4seayn9QIZH14veKSADSde8YGAJ05bifctH9i+xqf7M boQenHfwgAsQbrjcJh8TCMzHFlg/8pKXo6Pop1wg52KA/4WzsumKD9UX/z5lGAl1at8N fZSjr1urTbolfCgcRKbvKHogssam5CAO+oDpproJS6WI3SQVW5spRCP1ePUuooGDrqXN BKuqL3exOkwpCiMXvvi+n1PiFAq501/SW9bM+5Oe9hhkBjzR/1jJNeqsqVG8JPEu99HY CTOhlw0zfbuS1jZcOFJaYBg0pTG+vQoNHyWjht+bVCpqZV3FiuqHN7kT8n8t+LB/PYXb 6cuQ== X-Gm-Message-State: ALyK8tJV1p1JlAgAB38AkJzzF/PJxW8l+N7LhPSFP7HqE63f453TW51fEaNYZbBRKHYHb0SoocH5h92hPtlQuw== MIME-Version: 1.0 X-Received: by 10.107.48.203 with SMTP id w194mr2390958iow.123.1464930328822; Thu, 02 Jun 2016 22:05:28 -0700 (PDT) Received: by 10.36.113.3 with HTTP; Thu, 2 Jun 2016 22:05:28 -0700 (PDT) In-Reply-To: References: Date: Thu, 2 Jun 2016 22:05:28 -0700 Message-ID: Subject: Re: CLOCK_MONOTONIC / CLOCK_UPTIME is not really monotonic between threads From: Adrian Chadd To: Maxim Sobolev Cc: FreeBSD Current , freebsd-threads@freebsd.org, FreeBSD Questions , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Jun 2016 05:05:29 -0000 [snip] a) is it on one core, or multiple cores? b) CLOCK_MONOTONIC_FAST? c) is it on a system that /has/ invariant-TSC ? d) is this a multi-socket system? -adrian From owner-freebsd-threads@freebsd.org Fri Jun 3 10:09:48 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D01FB66AC4; Fri, 3 Jun 2016 10:09:48 +0000 (UTC) (envelope-from pieter@degoeje.nl) Received: from degoeje.nl (degoeje.nl [81.169.238.128]) by mx1.freebsd.org (Postfix) with ESMTP id 245B41878; Fri, 3 Jun 2016 10:09:47 +0000 (UTC) (envelope-from pieter@degoeje.nl) Received: from [192.168.1.250] (unknown [188.203.228.182]) by degoeje.nl (Postfix) with ESMTPSA id 1BD4115C0025; Fri, 3 Jun 2016 12:09:39 +0200 (CEST) Subject: Re: CLOCK_MONOTONIC / CLOCK_UPTIME is not really monotonic between threads To: Maxim Sobolev , FreeBSD Current , freebsd-threads@freebsd.org, freebsd-questions@freebsd.org, freebsd-arch@freebsd.org References: From: Pieter de Goeje Message-ID: Date: Fri, 3 Jun 2016 12:09:38 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=3.5 tests=ALL_TRUSTED autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on degoeje.nl X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Jun 2016 10:09:48 -0000 Op 2016-06-03 om 06:16 schreef Maxim Sobolev: > 4. If the answer for (3) is yes, then what is the method to disable using > TSC and use slower but possibly more reliable method? sysctl kern.timecounter.choice lists the available timers. sysctl kern.timecounter.hardware selects the timer. Changes take effect immediately. - Pieter From owner-freebsd-threads@freebsd.org Fri Jun 3 14:28:33 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D2B9B6856D for ; Fri, 3 Jun 2016 14:28:33 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: from mail-oi0-x232.google.com (mail-oi0-x232.google.com [IPv6:2607:f8b0:4003:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 110171AAF for ; Fri, 3 Jun 2016 14:28:33 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: by mail-oi0-x232.google.com with SMTP id w184so129128153oiw.2 for ; Fri, 03 Jun 2016 07:28:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sippysoft-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=/R1r4y9qtKUG1dHzy0HWyVDbah0Oa/cO79CxNqLFass=; b=x279AAiC/I/Tnt/vlSykudYqoId3KDJnLN5N64dcBjybxaYicygHPAUQKlfU+XhtSp aYhkB/ZbVO8Ipc+u4kIANOl1fMs3QZH0MrGYpVzL5runJ5AF/uC7UD98TDg8+L95CwAA YlF+4Lru9l1OSOEmCXYgdE/MwWK3O5IQJ3weBYI3dKpwEiz1Kfvf+guniqyEAnBsrgs5 x+unrPYfDeNk4BQ+RQf+yYb9y1fB2PaAkRxmjgZjHxvDXZBn3qi0oVH+uhXbeYoWhjaz BgNiW/60SjOysNd8EljJEqG6myMfTxgiFcxYz2nkLXEQlwYmfVySzUspF9B+m/zc625O ga2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=/R1r4y9qtKUG1dHzy0HWyVDbah0Oa/cO79CxNqLFass=; b=FNCNV12J44FYXni75Vf7TrwX58/NorojAF9/EIClLyFyTQse5DkBmjBH+Aiebr9bez P7cFkLzLKX8etJ/FgTZ9OKjAseBMbANZgJvoMM5YcYuqg4EEmzJrTwxnDj3V6W5/hqwL AZSgbVr6EYAded/Xpu4V9nxK4XXRxH61U4uex3zM8N3rNdPaTL3S+wowi/Nr6c1GieHK oqSCmCNoRnuaTb51aMfiBiE5MF3NFVZDfaRy8zajKzPzJdOzorKWewSfi8MlYukoR6A+ LPQZdYDHDdfFxB9bmDseo4gDbhU+fNc3l3/aWJzMse5HIevcP7DxgoMWmRxevvkg2QL3 qGpw== X-Gm-Message-State: ALyK8tJutal/ghp/j6D5EhjUicuZfRzIvluew3yIQXNko565qj/l52q39EMU5GIb75hywgSHtD4GAIf+0YBUyQ3t X-Received: by 10.157.31.78 with SMTP id x14mr2143800otx.15.1464964112288; Fri, 03 Jun 2016 07:28:32 -0700 (PDT) MIME-Version: 1.0 Sender: sobomax@sippysoft.com Received: by 10.157.56.70 with HTTP; Fri, 3 Jun 2016 07:28:31 -0700 (PDT) In-Reply-To: References: From: Maxim Sobolev Date: Fri, 3 Jun 2016 07:28:31 -0700 X-Google-Sender-Auth: _Ht2eSLKxKB2HEHmY-rA1U04kUI Message-ID: Subject: Re: CLOCK_MONOTONIC / CLOCK_UPTIME is not really monotonic between threads To: Adrian Chadd Cc: FreeBSD Current , freebsd-threads@freebsd.org, FreeBSD Questions , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Jun 2016 14:28:33 -0000 a. multiple cores. b. makes no difference c. yes, I believe so kern.timecounter.tsc_shift: 1 kern.timecounter.smp_tsc_adjust: 0 kern.timecounter.smp_tsc: 1 kern.timecounter.invariant_tsc: 1 machdep.tsc_freq: 2658118740 machdep.disable_tsc_calibration: 0 machdep.disable_tsc: 0 d. no, single socket Intel Q6700. I've also seen this problem on core i7-4770 running virtualbox 5.x. I have a hints that this also happens on our bigger production boxes, but I have no specifics yet. On Thu, Jun 2, 2016 at 10:05 PM, Adrian Chadd wrote: > [snip] > > a) is it on one core, or multiple cores? > b) CLOCK_MONOTONIC_FAST? > c) is it on a system that /has/ invariant-TSC ? > d) is this a multi-socket system? > > > > -adrian > > From owner-freebsd-threads@freebsd.org Sat Jun 4 11:09:58 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A5B0AB69F68; Sat, 4 Jun 2016 11:09:58 +0000 (UTC) (envelope-from seotownsend@icloud.com) Received: from st11p01im-asmtp001.me.com (st11p01im-asmtp001.me.com [17.172.204.151]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 78EFD1E55; Sat, 4 Jun 2016 11:09:58 +0000 (UTC) (envelope-from seotownsend@icloud.com) Received: from process-dkim-sign-daemon.st11p01im-asmtp001.me.com by st11p01im-asmtp001.me.com (Oracle Communications Messaging Server 7.0.5.36.0 64bit (built Sep 8 2015)) id <0O8801E00S7A6Q00@st11p01im-asmtp001.me.com>; Sat, 04 Jun 2016 10:09:48 +0000 (GMT) Received: from [10.30.1.6] (unknown [104.156.228.75]) by st11p01im-asmtp001.me.com (Oracle Communications Messaging Server 7.0.5.36.0 64bit (built Sep 8 2015)) with ESMTPSA id <0O8800DJDS891630@st11p01im-asmtp001.me.com>; Sat, 04 Jun 2016 10:09:47 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-06-04_03:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1011 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1510270003 definitions=main-1606040116 MIME-version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: CLOCK_MONOTONIC / CLOCK_UPTIME is not really monotonic between threads From: Seo Townsend In-reply-to: Date: Sat, 04 Jun 2016 03:09:44 -0700 Cc: Adrian Chadd , "freebsd-arch@freebsd.org" , FreeBSD Current , FreeBSD Questions , freebsd-threads@freebsd.org Message-id: References: To: Maxim Sobolev X-Mailer: Apple Mail (2.3124) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=icloud.com; s=4d515a; t=1465034988; bh=kLj1bwabNtxVLQmtlFkraJhlhovJ1iVjbxBt0eUnpiU=; h=Content-type:MIME-version:Subject:From:Date:Message-id:To; b=BvOWAs70DrcIGAR4HE1Bck50E5kbedqOPuAkoGMsSOrWiHcaQKTIuLR9wcXRimCwt S0W2cr+PfTJKropgJhMLiW54rEB5DC/34vreIJWraRyTOlP3BLkrwmlp0lssNw7RNy Qm7W9uWtklW+TXvZgCVeO0gmxxx5GA+27DkjjAzBHzE+z1SlVymS1CpelpUi7bIFFb qfrhmZXyOjvJMmcV9tBOjDZ3jy7i/4k2kv4sJmVgkYlHw3BWdEREuPgyN3xBvBpl3d RB6TdubyzFECYKgLzo8I1P9XSO93midPOnlntnlRoL972ivqvhuJL9RucHWITf+I03 0djXGb0vSDquQ== Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Jun 2016 11:09:58 -0000 Heyy, Sure your machine isn=E2=80=99t sitting next to a block hole? :P Many of the names (CLOCK_XXXXXX) seem to be just aliased. https://github.com/freebsd/freebsd/blob/master/sys/kern/kern_time.c#L343 = = I didn=E2=80=99t thoroughly look at this, but it seems like there=E2=80=99= s a margin of noise introduced by drift corrections that might (half ass-edly) explain the = sub millisecond errors you were receiving? https://github.com/freebsd/freebsd/blob/master/sys/kern/kern_tc.c#L1374 = For the larger >1ms errors, ...that=E2=80=99s probably not good? lol. = Might be worth it to try the timersub library function: = (http://www.unix.com/man-page/freebsd/3/timercmp/) to calculate the delta. > On Jun 3, 2016, at 7:28 AM, Maxim Sobolev wrote: >=20 > a. multiple cores. > b. makes no difference > c. yes, I believe so >=20 > kern.timecounter.tsc_shift: 1 > kern.timecounter.smp_tsc_adjust: 0 > kern.timecounter.smp_tsc: 1 > kern.timecounter.invariant_tsc: 1 > machdep.tsc_freq: 2658118740 > machdep.disable_tsc_calibration: 0 > machdep.disable_tsc: 0 >=20 > d. no, single socket Intel Q6700. I've also seen this problem on core > i7-4770 running virtualbox 5.x. I have a hints that this also happens = on > our bigger production boxes, but I have no specifics yet. >=20 > On Thu, Jun 2, 2016 at 10:05 PM, Adrian Chadd > wrote: >=20 >> [snip] >>=20 >> a) is it on one core, or multiple cores? >> b) CLOCK_MONOTONIC_FAST? >> c) is it on a system that /has/ invariant-TSC ? >> d) is this a multi-socket system? >>=20 >>=20 >>=20 >> -adrian >>=20 >>=20 > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org"