From owner-freebsd-hackers@freebsd.org Sat Dec 10 19:52:59 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03367C7074C; Sat, 10 Dec 2016 19:52:59 +0000 (UTC) (envelope-from kargl@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D326F1D75; Sat, 10 Dec 2016 19:52:58 +0000 (UTC) (envelope-from kargl@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.15.2/8.15.2) with ESMTPS id uBAJqvh0024797 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 10 Dec 2016 11:52:57 -0800 (PST) (envelope-from kargl@troutmask.apl.washington.edu) Received: (from kargl@localhost) by troutmask.apl.washington.edu (8.15.2/8.15.2/Submit) id uBAJquL7024796; Sat, 10 Dec 2016 11:52:56 -0800 (PST) (envelope-from kargl) Date: Sat, 10 Dec 2016 11:52:56 -0800 From: "Steven G. Kargl" To: Hans Petter Selasky Cc: kargl@uw.edu, freebsd-current@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: System hangs at boot in xhci0 Message-ID: <20161210195256.GA24787@troutmask.apl.washington.edu> Reply-To: kargl@uw.edu References: <20161209210940.GA1144@troutmask.apl.washington.edu> <5a509678-2a67-b538-ea42-d9e80a6bd3eb@selasky.org> <20161209234530.GA21666@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161209234530.GA21666@troutmask.apl.washington.edu> User-Agent: Mutt/1.6.1 (2016-04-27) X-Mailman-Approved-At: Sun, 11 Dec 2016 05:23:01 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Dec 2016 19:52:59 -0000 On Fri, Dec 09, 2016 at 03:45:30PM -0800, Steven G. Kargl wrote: > On Fri, Dec 09, 2016 at 11:58:21PM +0100, Hans Petter Selasky wrote: > > On 12/09/16 22:09, Steve Kargl wrote: > > > I updated my system to > > > > > > % svn info /usr/src > > > Revision: 309748 > > > > > > Built a shiny new kernel, which hangs during boot. > > > There is no panic. Using the dmesg from kernel.old/kernel, > > > the last few reported are > > > > > > > > > pci2: on pcib2 > > > xhci0: mem 0xfe900000-0xfe900fff > > > irq 48 at device 0.0 on pci2 > > > xhci0: 32 bytes context size, 64-bit DMA > > > > > > At this point, the system is completely unresponse and > > > needs to be power cycled. > > > > > > > Hi, > > > > What is the next message in the old kernel which is printed? There has > > been zero changes in the XHCI driver recently. > > > > Can you copy /boot/kernel.old to /boot/kernel.works > > > > Then add this option to the GENERIC kernel config: > > > > options VERBOSE_SYSINIT > > > > What are the last few messages in dmesg when you boot with the above flag? > > > > With a boot_verbose of the new kernel I get the following output: > > xhci0: 32 bytes context size, 64-bit DMA > xhci0: attempting to allocate 1 MSI vectors (4 supported) > msi: routing MSI IRQ 260 to local APIC 16 vector 55 > xhci0: using IRQ 260 for MSI > xhci0: MSI enabled > usbus0 on xhci0 > xhci0: usbpf: Attached > random: harvesting attach, 8 bytes (4 bits) from usbus0 > random: harvesting attach, 8 bytes (4 bits) from xhci0 > random: harvesting attach, 8 bytes (4 bits) from pci2 > random: harvesting attach, 8 bytes (4 bits) from pcib2 > > and then the system locks up. With the old kernel (circa > Oct 10th sources), next few lines from dmesg are > > pcib3: irq 54 at device 10.0 on pci0 > pcib0: allocated type 4 (0xd000-0xdfff) for rid 1c of pcib3 > pcib0: allocated type 3 (0xfe800000-0xfe8fffff) for rid 20 of pcib3 > pcib3: domain 0 > pcib3: secondary bus 3 > pcib3: subordinate bus 3 > pcib3: I/O decode 0xd000-0xdfff > pcib3: memory decode 0xfe800000-0xfe8fffff > pci3: on pcib3 > > I think that hang isn't caused by xhci, but rather is a > victim on being the last successfully probed device. In the last > weeks there have been a few commits (309588, 309400, and 308953) > that touched ACPI. I'm currently reverting these changes to test > if one is causing the problem. > > I did see that one of these revisions specific mentions the > ALASKA AMI bios, which I happen to have. However, that commit > also mentions a skylake processor while I have an AMD FX-8350. > Well, I reverted each of the above indicated revisions. No luck. New kernel still locks up. I guess it's time to do a bisection. Unfortunately, this cross a llvm 3.8 to 3.9 boundary. :( -- Steve http://troutmask.apl.washington.edu/~kargl/ https://www.youtube.com/watch?v=6hwgPfCcpyQ From owner-freebsd-hackers@freebsd.org Sun Dec 11 12:46:41 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B9619C705D9 for ; Sun, 11 Dec 2016 12:46:41 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mailout.stack.nl (mailout05.stack.nl [IPv6:2001:610:1108:5010::202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailout.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 849F71933 for ; Sun, 11 Dec 2016 12:46:41 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mailout.stack.nl (Postfix) with ESMTP id 2DDBC41F5; Sun, 11 Dec 2016 13:46:38 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id 1AA1028494; Sun, 11 Dec 2016 13:46:38 +0100 (CET) Date: Sun, 11 Dec 2016 13:46:38 +0100 From: Jilles Tjoelker To: Jonathan de Boyne Pollard Cc: FreeBSD Hackers , supervision@list.skarnet.org Subject: Re: NOTE_TRACK, EVFILT_PROC, kqueue, and subreapers Message-ID: <20161211124637.GA17315@stack.nl> References: <20161201172846.GP3428@protected.rcdrun.com> <20161201124118.46778e2b@mydesk.domain.cxm> <20161201174837.GR3428@protected.rcdrun.com> <20161201125438.15230317@mydesk.domain.cxm> <20161206104020.6b2ebb30@eto-mona.office.smartweb.sk> <20161206102637.1ddd152a@mydesk.domain.cxm> <20161207155638.4b2dd629@eto-mona.office.smartweb.sk> <630ace89-e29b-d0d3-9f15-110d8dc3de08@NTLWorld.com> <20161208132842.5d7940bd@eto-mona.office.smartweb.sk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 12:46:41 -0000 On Thu, Dec 08, 2016 at 08:23:29PM +0000, Jonathan de Boyne Pollard wrote: > Martin "eto" Misuth: > > I think that might be the reason why my PID1 s6-svscan on FreeBSD is > > accumulating zombies sometimes (seems like it is affected by dead > > descendants of ssh and my experiments). [...] > > Anyway as you are probably much closer to FreeBSD team than I am, [...] > I'm not. You have the same access as I and the rest of the world have. > For what it's worth, I've seen similar behaviour with zombies lying > around. If we can nail it down you can file a kernel bug report. > Have you checked that you aren't getting a NOTE_EXIT? As reported in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=213928 reapers do not receive SIGCHLD when inheriting zombies, although they do receive SIGCHLD when an inherited descendant process later terminates and are awakened when in a wait call for a matching process. This should probably be fixed. NOTE_TRACK has the inherent problem of what to do when NOTE_TRACKERR happens and has problems with process ID reuse, so it seems better to use reaper instead of it. -- Jilles Tjoelker From owner-freebsd-hackers@freebsd.org Sun Dec 11 17:54:57 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 90863C72984 for ; Sun, 11 Dec 2016 17:54:57 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 05AF21A6A for ; Sun, 11 Dec 2016 17:54:56 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBBHspX8072319 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 11 Dec 2016 19:54:52 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBBHspX8072319 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBBHspDu072318; Sun, 11 Dec 2016 19:54:51 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 11 Dec 2016 19:54:51 +0200 From: Konstantin Belousov To: Jilles Tjoelker Cc: Jonathan de Boyne Pollard , FreeBSD Hackers , supervision@list.skarnet.org Subject: Re: NOTE_TRACK, EVFILT_PROC, kqueue, and subreapers Message-ID: <20161211175451.GV54029@kib.kiev.ua> References: <20161201124118.46778e2b@mydesk.domain.cxm> <20161201174837.GR3428@protected.rcdrun.com> <20161201125438.15230317@mydesk.domain.cxm> <20161206104020.6b2ebb30@eto-mona.office.smartweb.sk> <20161206102637.1ddd152a@mydesk.domain.cxm> <20161207155638.4b2dd629@eto-mona.office.smartweb.sk> <630ace89-e29b-d0d3-9f15-110d8dc3de08@NTLWorld.com> <20161208132842.5d7940bd@eto-mona.office.smartweb.sk> <20161211124637.GA17315@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161211124637.GA17315@stack.nl> User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 17:54:57 -0000 On Sun, Dec 11, 2016 at 01:46:38PM +0100, Jilles Tjoelker wrote: > As reported in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=213928 > reapers do not receive SIGCHLD when inheriting zombies, although they do > receive SIGCHLD when an inherited descendant process later terminates > and are awakened when in a wait call for a matching process. This should > probably be fixed. I agree, in principle. Could you update your tests/sys/kern/reaper.c to include this scenario ? Untested change to the kernel side is below. diff --git a/sys/kern/kern_exit.c b/sys/kern/kern_exit.c index f4f453c3556..92899740eef 100644 --- a/sys/kern/kern_exit.c +++ b/sys/kern/kern_exit.c @@ -455,6 +455,12 @@ exit1(struct thread *td, int rval, int signo) if (!(q->p_flag & P_TRACED)) { proc_reparent(q, q->p_reaper); + if (q->p_state == PRS_ZOMBIE) { + PROC_LOCK(q->p_reaper); + pksignal(q->p_reaper, SIGCHLD, q->p_ksi); + wakeup(q->p_reaper); + PROC_UNLOCK(q->p_reaper); + } } else { /* * Traced processes are killed since their existence From owner-freebsd-hackers@freebsd.org Sun Dec 11 19:36:04 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 113CCC72483 for ; Sun, 11 Dec 2016 19:36:04 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id E86BD173E for ; Sun, 11 Dec 2016 19:36:03 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: by mailman.ysv.freebsd.org (Postfix) id E4D97C72482; Sun, 11 Dec 2016 19:36:03 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E473DC72481 for ; Sun, 11 Dec 2016 19:36:03 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: from mail-yw0-x232.google.com (mail-yw0-x232.google.com [IPv6:2607:f8b0:4002:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B0D8C173D for ; Sun, 11 Dec 2016 19:36:03 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: by mail-yw0-x232.google.com with SMTP id i145so50676399ywg.2 for ; Sun, 11 Dec 2016 11:36:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuxi-nl.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=hiYs9LGpLt9pJ51fddYS5023QoBMXOlbD4fOOqw50Fs=; b=u+DnMDJvA31NQXPddNRkbTV66xbx6yTjWnNJEo5vWEDoaatVSZzlpqZnWaJE3DOmHU lYCFvQDXGncNL/sLR1k3Mlj44WvghadNjA3heBcjjzZCiG2Q9hEelNyLawM33FeXA9+i btg7YYxpot5+lBhnadN8LhKslpH5z7KyrK2+wY1GM039GUgPVZ76FCi9zwzgkZt5Bh4c 5Lw8eN6DJYOcNT+hbN7yGSoYflk7lRTq1qVKQo33HL6vn7FDLtW1DDH4HsoPyXRz6ApA VUOpdh4unNlVL1Cx1MWjCgYPjqAiYrnty6y0IBGsGnkcWYXMV6G8oIfQas+WERR8iRv7 c3Cw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=hiYs9LGpLt9pJ51fddYS5023QoBMXOlbD4fOOqw50Fs=; b=EtQVT1e4feLhmLWi099Z5u1OucEiAYIhhSu5atqc9namj1v+WAKQvfo9eks4sGIdib Oq8r8ISywflOTlJypKXa9m30DjmSpePX3dUM5biP4L25fKNAxFO1OuHBziegHq1kHn9W cKysl5n4Qup8cvr3cHdIGNpGbYLw90zzPHTTMtnQl+/YbUiN1lpSoLyHCAuxJ5Pspv9F fCHW1kjB/X9LZJVB27CtHmcYVUxlRIT7Chy9w2xprgCLtHMabONL33DNKxNynskxNHp0 ll6EzH76A+HZ5TyqIv/6P3SzeRdAtF5oDA3OgZYHi2Ff3anvVZmX1WN6mvHVa4anPXrj TpRg== X-Gm-Message-State: AKaTC03AO4JAfb2CeXTyXskBSiveBbHzFb8UeqNKFkxXIkfS6PG6Pxb6Wycdc5PuXtYhsyoG9aisfclWf7VwBw== X-Received: by 10.129.164.198 with SMTP id b189mr80534015ywh.294.1481484962535; Sun, 11 Dec 2016 11:36:02 -0800 (PST) MIME-Version: 1.0 Received: by 10.129.0.212 with HTTP; Sun, 11 Dec 2016 11:35:32 -0800 (PST) From: Ed Schouten Date: Sun, 11 Dec 2016 20:35:32 +0100 Message-ID: Subject: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 19:36:04 -0000 Hi there, The last couple of months I've been playing around with a monitoring system called Prometheus (https://prometheus.io/). In short, Prometheus works like this: ----- If you already know Prometheus, skip this ----- 1. For the thing you want to monitor, you either integrate the Prometheus client library into your codebase or you write a separate exporter process. The client library or the exporter process then exposes key metrics of your application over HTTP. Simplified example: $ curl http://localhost:12345/metrics # HELP open_file_descriptors The number of files opened by the process # TYPE open_file_descriptors gauge open_file_descriptors 12 # HELP http_requests The number of HTTP requests received. # TYPE http_requests counter http_requests{result="2xx"} 100 http_requests{result="4xx"} 14 http_requests{result="5xx"} 0 2. You fire op Prometheus and configure it to scrape and store all of the things you want to monitor. Prometheus can then add more labels to the metrics it scrapes. So the example above may get transformed by Prometheus to look like this: open_file_descriptors{job="nginx",instance="web1.mycompany.com"} 12 http_requests{job="nginx",instance="web1.mycompany.com",result="2xx"} 100 http_requests{job="nginx",instance="web1.mycompany.com",result="4xx"} 14 http_requests{job="nginx",instance="web1.mycompany.com",result="5xx"} 0 Fun fact: Prometheus can also scrape Prometheus, so if you operate multiple datacenters, you can let a global instance scrape a per-DC instance and add a dc="..." label to all metrics. 3. After scraping data for some time, you can do fancy queries like these: - Compute the 5-minute rate of HTTP requests per server and per HTTP error code: rate(http_requests[5m]) - Compute the 5-minute rate of all HTTP requests on the entire cluster: sum(rate(http_requests[5m])) - Same as the above, but aggregate by HTTP error code: sum(rate(http_requests[5m])) by (result) Prometheus can do alerting as well by using these expressions as matchers. 4. Set up Grafana and voila: you can create fancy dashboards! ----- If you skipped the introduction, start reading here ----- The Prometheus folks have developed a tool called the node_exporter (https://github.com/prometheus/node_exporter). Basically it extracts a whole bunch of interesting system-related metrics (disk usage, network I/O, etc) through sysctl(3), invoking ioctl(2), parsing /proc files, etc. and exposes that information using Prometheus' syntax. The other day I was thinking: in a certain way, the node exporter is a bit of a redundant tool on the BSDs. Instead of needing to write custom collectors for every kernel subsystem, we could write a generic exporter for converting the entire sysctl(3) tree to Prometheus metrics, which is exactly what I'm experimenting with here: https://github.com/EdSchouten/prometheus_sysctl_exporter An example of what this tool's output looks like: $ ./prometheus_sysctl_exporter ... # HELP kern_maxfiles Maximum number of files sysctl_kern_maxfiles 1043382 # HELP kern_openfiles System-wide number of open files sysctl_kern_openfiles 316 ... You could use this to write alerting rules like this: ALERT FileDescriptorUsageHigh IF sysctl_kern_openfiles / sysctl_kern_maxfiles > 0.5 FOR 15m ANNOTATIONS { description = "More than half of all FDs are in use!", } There you go. Access to a very large number of metrics without too much effort. My main question here is: are there any people in here interested in seeing something like this being developed into something usable? If so, let me know and I'll pursue this further. I also have a couple of technical questions related to sysctl(3)'s in-kernel design: - Prometheus differentiates between gauges (memory usage), counters (number of HTTP requests), histograms (per-RPC latency stats), etc., while sysctl(3) does not. It would be nice if we could have that info on a per-sysctl basis. Mind if I add a CTLFLAG_GAUGE, CTLFLAG_COUNTER, etc? - Semantically sysctl(3) and Prometheus are slightly different. Consider this sysctl: hw.acpi.thermal.tz0.temperature: 27.8C My tool currently converts this metric's name to sysctl_hw_acpi_thermal_tz0_temperature. This is suboptimal, as it would ideally be called sysctl_hw_acpi_thermal_temperature{sensor="tz0"}. Otherwise you wouldn't be able to write generic alerting rules, use aggregation in queries, etc. I was thinking: we could quite easily do such a translation by attaching labels to SYSCTL_NODE objects. As in, the hw.acpi.thermal node would get a label "sensor". Any OID placed underneath this node will not become a midfix of the sysctl name, but the value of that label instead. Thoughts? A final remark I want to make: a concern might be that changes like these would not be generic, but only apply to Prometheus. I tend to disagree. First of all, an advantage of Prometheus is that the coupling is very loose: it's just a GET request with key-value pairs. Anyone is free to add his/her own implementation. Second, emaste@ also pointed me to another monitoring framework being developed by Intel right now: https://github.com/intelsdi-x/snap The changes I'm proposing would also seem to make exporting sysctl data to that system easier. Anyway, thanks for reading this huge wall of text. Best regards, -- Ed Schouten Nuxi, 's-Hertogenbosch, the Netherlands KvK-nr.: 62051717 From owner-freebsd-hackers@freebsd.org Sun Dec 11 19:55:37 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE025C72AB3 for ; Sun, 11 Dec 2016 19:55:37 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C1B28138F for ; Sun, 11 Dec 2016 19:55:37 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (unknown [10.1.1.2]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 38156DC57 for ; Sun, 11 Dec 2016 19:55:36 +0000 (UTC) Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: freebsd-hackers@freebsd.org References: From: Allan Jude Message-ID: Date: Sun, 11 Dec 2016 14:55:35 -0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 19:55:38 -0000 On 2016-12-11 14:35, Ed Schouten wrote: > Hi there, > > The last couple of months I've been playing around with a monitoring > system called Prometheus (https://prometheus.io/). In short, > Prometheus works like this: > > ----- If you already know Prometheus, skip this ----- > > 1. For the thing you want to monitor, you either integrate the > Prometheus client library into your codebase or you write a separate > exporter process. The client library or the exporter process then > exposes key metrics of your application over HTTP. Simplified example: > > $ curl http://localhost:12345/metrics > # HELP open_file_descriptors The number of files opened by the process > # TYPE open_file_descriptors gauge > open_file_descriptors 12 > # HELP http_requests The number of HTTP requests received. > # TYPE http_requests counter > http_requests{result="2xx"} 100 > http_requests{result="4xx"} 14 > http_requests{result="5xx"} 0 > > 2. You fire op Prometheus and configure it to scrape and store all of > the things you want to monitor. Prometheus can then add more labels to > the metrics it scrapes. So the example above may get transformed by > Prometheus to look like this: > > open_file_descriptors{job="nginx",instance="web1.mycompany.com"} 12 > http_requests{job="nginx",instance="web1.mycompany.com",result="2xx"} 100 > http_requests{job="nginx",instance="web1.mycompany.com",result="4xx"} 14 > http_requests{job="nginx",instance="web1.mycompany.com",result="5xx"} 0 > > Fun fact: Prometheus can also scrape Prometheus, so if you operate > multiple datacenters, you can let a global instance scrape a per-DC > instance and add a dc="..." label to all metrics. > > 3. After scraping data for some time, you can do fancy queries like these: > > - Compute the 5-minute rate of HTTP requests per server and per HTTP error code: > rate(http_requests[5m]) > > - Compute the 5-minute rate of all HTTP requests on the entire cluster: > sum(rate(http_requests[5m])) > > - Same as the above, but aggregate by HTTP error code: > sum(rate(http_requests[5m])) by (result) > > Prometheus can do alerting as well by using these expressions as matchers. > > 4. Set up Grafana and voila: you can create fancy dashboards! > > ----- If you skipped the introduction, start reading here ----- > > The Prometheus folks have developed a tool called the node_exporter > (https://github.com/prometheus/node_exporter). Basically it extracts a > whole bunch of interesting system-related metrics (disk usage, network > I/O, etc) through sysctl(3), invoking ioctl(2), parsing /proc files, > etc. and exposes that information using Prometheus' syntax. > > The other day I was thinking: in a certain way, the node exporter is a > bit of a redundant tool on the BSDs. Instead of needing to write > custom collectors for every kernel subsystem, we could write a generic > exporter for converting the entire sysctl(3) tree to Prometheus > metrics, which is exactly what I'm experimenting with here: > > https://github.com/EdSchouten/prometheus_sysctl_exporter > > An example of what this tool's output looks like: > > $ ./prometheus_sysctl_exporter > ... > # HELP kern_maxfiles Maximum number of files > sysctl_kern_maxfiles 1043382 > # HELP kern_openfiles System-wide number of open files > sysctl_kern_openfiles 316 > ... > > You could use this to write alerting rules like this: > > ALERT FileDescriptorUsageHigh > IF sysctl_kern_openfiles / sysctl_kern_maxfiles > 0.5 > FOR 15m > ANNOTATIONS { > description = "More than half of all FDs are in use!", > } > > There you go. Access to a very large number of metrics without too much effort. > > My main question here is: are there any people in here interested in > seeing something like this being developed into something usable? If > so, let me know and I'll pursue this further. > > I also have a couple of technical questions related to sysctl(3)'s > in-kernel design: > > - Prometheus differentiates between gauges (memory usage), counters > (number of HTTP requests), histograms (per-RPC latency stats), etc., > while sysctl(3) does not. It would be nice if we could have that info > on a per-sysctl basis. Mind if I add a CTLFLAG_GAUGE, CTLFLAG_COUNTER, > etc? > > - Semantically sysctl(3) and Prometheus are slightly different. > Consider this sysctl: > > hw.acpi.thermal.tz0.temperature: 27.8C > > My tool currently converts this metric's name to > sysctl_hw_acpi_thermal_tz0_temperature. This is suboptimal, as it > would ideally be called > sysctl_hw_acpi_thermal_temperature{sensor="tz0"}. Otherwise you > wouldn't be able to write generic alerting rules, use aggregation in > queries, etc. > > I was thinking: we could quite easily do such a translation by > attaching labels to SYSCTL_NODE objects. As in, the hw.acpi.thermal > node would get a label "sensor". Any OID placed underneath this node > will not become a midfix of the sysctl name, but the value of that > label instead. Thoughts? > > A final remark I want to make: a concern might be that changes like > these would not be generic, but only apply to Prometheus. I tend to > disagree. First of all, an advantage of Prometheus is that the > coupling is very loose: it's just a GET request with key-value pairs. > Anyone is free to add his/her own implementation. > > Second, emaste@ also pointed me to another monitoring framework being > developed by Intel right now: > > https://github.com/intelsdi-x/snap > > The changes I'm proposing would also seem to make exporting sysctl > data to that system easier. > > Anyway, thanks for reading this huge wall of text. > > Best regards, > I would find this very useful. I've wanted to have something like this, and expose more data (especially re: disks) via sysctl for a long time. The nonsense we do to deal with monitoring dev.cpu.[0-n].temperature and deal with it are rather annoying. I like the sound of everything you propose. Especially flagging sysctls so it is possible to programmatically get 'type' information. -- Allan Jude From owner-freebsd-hackers@freebsd.org Sun Dec 11 20:07:59 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C285EC72FC9 for ; Sun, 11 Dec 2016 20:07:59 +0000 (UTC) (envelope-from mozolevsky@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 9FB3D1F50 for ; Sun, 11 Dec 2016 20:07:59 +0000 (UTC) (envelope-from mozolevsky@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 9EFD7C72FC8; Sun, 11 Dec 2016 20:07:59 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E941C72FC6 for ; Sun, 11 Dec 2016 20:07:59 +0000 (UTC) (envelope-from mozolevsky@gmail.com) Received: from mail-wj0-x22c.google.com (mail-wj0-x22c.google.com [IPv6:2a00:1450:400c:c01::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 268311F4F for ; Sun, 11 Dec 2016 20:07:59 +0000 (UTC) (envelope-from mozolevsky@gmail.com) Received: by mail-wj0-x22c.google.com with SMTP id tk12so55928011wjb.3 for ; Sun, 11 Dec 2016 12:07:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=TCeVnHlDx5+SI2+qi6N9U3ITGSqvJpfJskWyga19S1E=; b=i82mKsTofkTN5zNBeVml7VDxXwJY/iA43oDDL5XQk7M2xb3vIRewpYju66edDnQz42 JSkGxqeagSeWzvsY1iNY3GiKXLBkV+9hlF/4Y0olmzR4PdnZAERbzq5lXfBVbEGCzl+/ wkgHSbx12DxfedVcc2bdrg/7lQAg3Hn1ZzDGGdAVxKe83sk5tT+CROBHdY2kacRYN6ze Xd3e0NvoH0kAf8X+GF4IEGpBGPTdV0DKBQJEfgwYJvZUIDkA9AffENXeeLqbqqdqHjSY ppnpuN0BR1LCj0TjJo+Ji31z/8Qi7yTQ0jt7kD3asost2C4jwFFpPxmzD7Ceujd85McY G/ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=TCeVnHlDx5+SI2+qi6N9U3ITGSqvJpfJskWyga19S1E=; b=Hx57IkLGDnuB3flN1oTxzEijHYlXOi+Iby5eMiVa+3UHU/PpUoAvHj0xLaVpIu1vpK UPshJHdxV43hITc4GoZsRwiD7bJCOHcrFzhJCCnB562g+MoosBso92ZTIjZ3+Ik2N7Ik KPvIR6UQe1DosCoFeQv0z4kQWqJqCFg4YRKP1nyeeMmwv6PscEnwuYqNZBfNf4hZ+mZZ fYVIaLTSHHU0gRz5+kJMXtNh9AWuMvZ+9Ki7sjqJgwVpE+Om0pbmTgGVAStvS4+4VM1F kL2iQw+sr/9Rb94FkxSqw8i+dWgXeJ6L8nwrYMeP9jwO5tv+kFnu2rlDCchzfKcb4wTd 1Naw== X-Gm-Message-State: AKaTC03bczCg3mRORL17+yWzlj52Kw8qlfegjGCcPpNz3haPyymd+5J8F/eZdZ/2fLL8pq0XAFBepM38U14eZg== X-Received: by 10.194.85.77 with SMTP id f13mr77427124wjz.187.1481486877081; Sun, 11 Dec 2016 12:07:57 -0800 (PST) MIME-Version: 1.0 Sender: mozolevsky@gmail.com Received: by 10.28.69.88 with HTTP; Sun, 11 Dec 2016 12:07:16 -0800 (PST) In-Reply-To: References: From: Igor Mozolevsky Date: Sun, 11 Dec 2016 20:07:16 +0000 X-Google-Sender-Auth: Im64ikZQFUybZiacjH7CWHIRK80 Message-ID: Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: Ed Schouten Cc: hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 20:07:59 -0000 Can SNMP not do that already? -- Igor M. On 11 December 2016 at 19:35, Ed Schouten wrote: > Hi there, > > The last couple of months I've been playing around with a monitoring > system called Prometheus (https://prometheus.io/). In short, > Prometheus works like this: > > ----- If you already know Prometheus, skip this ----- > > 1. For the thing you want to monitor, you either integrate the > Prometheus client library into your codebase or you write a separate > exporter process. The client library or the exporter process then > exposes key metrics of your application over HTTP. Simplified example: > > $ curl http://localhost:12345/metrics > # HELP open_file_descriptors The number of files opened by the process > # TYPE open_file_descriptors gauge > open_file_descriptors 12 > # HELP http_requests The number of HTTP requests received. > # TYPE http_requests counter > http_requests{result="2xx"} 100 > http_requests{result="4xx"} 14 > http_requests{result="5xx"} 0 > > 2. You fire op Prometheus and configure it to scrape and store all of > the things you want to monitor. Prometheus can then add more labels to > the metrics it scrapes. So the example above may get transformed by > Prometheus to look like this: > > open_file_descriptors{job="nginx",instance="web1.mycompany.com"} 12 > http_requests{job="nginx",instance="web1.mycompany.com",result="2xx"} 100 > http_requests{job="nginx",instance="web1.mycompany.com",result="4xx"} 14 > http_requests{job="nginx",instance="web1.mycompany.com",result="5xx"} 0 > > Fun fact: Prometheus can also scrape Prometheus, so if you operate > multiple datacenters, you can let a global instance scrape a per-DC > instance and add a dc="..." label to all metrics. > > 3. After scraping data for some time, you can do fancy queries like these: > > - Compute the 5-minute rate of HTTP requests per server and per HTTP error > code: > rate(http_requests[5m]) > > - Compute the 5-minute rate of all HTTP requests on the entire cluster: > sum(rate(http_requests[5m])) > > - Same as the above, but aggregate by HTTP error code: > sum(rate(http_requests[5m])) by (result) > > Prometheus can do alerting as well by using these expressions as matchers. > > 4. Set up Grafana and voila: you can create fancy dashboards! > > ----- If you skipped the introduction, start reading here ----- > > The Prometheus folks have developed a tool called the node_exporter > (https://github.com/prometheus/node_exporter). Basically it extracts a > whole bunch of interesting system-related metrics (disk usage, network > I/O, etc) through sysctl(3), invoking ioctl(2), parsing /proc files, > etc. and exposes that information using Prometheus' syntax. > > The other day I was thinking: in a certain way, the node exporter is a > bit of a redundant tool on the BSDs. Instead of needing to write > custom collectors for every kernel subsystem, we could write a generic > exporter for converting the entire sysctl(3) tree to Prometheus > metrics, which is exactly what I'm experimenting with here: > > https://github.com/EdSchouten/prometheus_sysctl_exporter > > An example of what this tool's output looks like: > > $ ./prometheus_sysctl_exporter > ... > # HELP kern_maxfiles Maximum number of files > sysctl_kern_maxfiles 1043382 > # HELP kern_openfiles System-wide number of open files > sysctl_kern_openfiles 316 > ... > > You could use this to write alerting rules like this: > > ALERT FileDescriptorUsageHigh > IF sysctl_kern_openfiles / sysctl_kern_maxfiles > 0.5 > FOR 15m > ANNOTATIONS { > description = "More than half of all FDs are in use!", > } > > There you go. Access to a very large number of metrics without too much > effort. > > My main question here is: are there any people in here interested in > seeing something like this being developed into something usable? If > so, let me know and I'll pursue this further. > > I also have a couple of technical questions related to sysctl(3)'s > in-kernel design: > > - Prometheus differentiates between gauges (memory usage), counters > (number of HTTP requests), histograms (per-RPC latency stats), etc., > while sysctl(3) does not. It would be nice if we could have that info > on a per-sysctl basis. Mind if I add a CTLFLAG_GAUGE, CTLFLAG_COUNTER, > etc? > > - Semantically sysctl(3) and Prometheus are slightly different. > Consider this sysctl: > > hw.acpi.thermal.tz0.temperature: 27.8C > > My tool currently converts this metric's name to > sysctl_hw_acpi_thermal_tz0_temperature. This is suboptimal, as it > would ideally be called > sysctl_hw_acpi_thermal_temperature{sensor="tz0"}. Otherwise you > wouldn't be able to write generic alerting rules, use aggregation in > queries, etc. > > I was thinking: we could quite easily do such a translation by > attaching labels to SYSCTL_NODE objects. As in, the hw.acpi.thermal > node would get a label "sensor". Any OID placed underneath this node > will not become a midfix of the sysctl name, but the value of that > label instead. Thoughts? > > A final remark I want to make: a concern might be that changes like > these would not be generic, but only apply to Prometheus. I tend to > disagree. First of all, an advantage of Prometheus is that the > coupling is very loose: it's just a GET request with key-value pairs. > Anyone is free to add his/her own implementation. > > Second, emaste@ also pointed me to another monitoring framework being > developed by Intel right now: > > https://github.com/intelsdi-x/snap > > The changes I'm proposing would also seem to make exporting sysctl > data to that system easier. > > Anyway, thanks for reading this huge wall of text. > > Best regards, > -- > Ed Schouten > Nuxi, 's-Hertogenbosch, the Netherlands > KvK-nr.: 62051717 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Sun Dec 11 20:12:37 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4365C722B5 for ; Sun, 11 Dec 2016 20:12:37 +0000 (UTC) (envelope-from jonathan@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C57EC6E8; Sun, 11 Dec 2016 20:12:37 +0000 (UTC) (envelope-from jonathan@FreeBSD.org) Received: from [192.168.42.150] (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) by freefall.freebsd.org (Postfix) with ESMTP id CB4F1744F; Sun, 11 Dec 2016 20:12:36 +0000 (UTC) (envelope-from jonathan@FreeBSD.org) From: "Jonathan Anderson" To: "Allan Jude" Cc: freebsd-hackers@freebsd.org Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems Date: Sun, 11 Dec 2016 16:42:38 -0330 Message-ID: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; format=flowed X-Mailer: MailMate (1.9.6r5310) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 20:12:38 -0000 On 11 Dec 2016, at 16:25, Allan Jude wrote: > On 2016-12-11 14:35, Ed Schouten wrote: >> >> The other day I was thinking: in a certain way, the node exporter is >> a >> bit of a redundant tool on the BSDs. Instead of needing to write >> custom collectors for every kernel subsystem, we could write a >> generic >> exporter for converting the entire sysctl(3) tree to Prometheus >> metrics [...] >> >> A final remark I want to make: a concern might be that changes like >> these would not be generic, but only apply to Prometheus. I tend to >> disagree. First of all, an advantage of Prometheus is that the >> coupling is very loose: it's just a GET request with key-value pairs. >> Anyone is free to add his/her own implementation. > > I would find this very useful. I've wanted to have something like > this, > and expose more data (especially re: disks) via sysctl for a long > time. > > The nonsense we do to deal with monitoring dev.cpu.[0-n].temperature > and > deal with it are rather annoying. > > I like the sound of everything you propose. Especially flagging > sysctls > so it is possible to programmatically get 'type' information. I, for one, agree, and at the risk of being slightly provocative, I think it could be taken even further. :) I'd love to be able to retrieve a sysctl entry or subtree as an nvlist_t. Rather than doing lots of system calls to enumerate clearly-related sysctl values, it would be very cool to do one system call and then inspect the resulting data structure in memory. We already use libnv in a number of places, and it seems like it could be a natural mapping from the sysctl hierarchy. Jon -- Jonathan Anderson jonathan@FreeBSD.org From owner-freebsd-hackers@freebsd.org Sun Dec 11 22:23:32 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D8B8C720DB for ; Sun, 11 Dec 2016 22:23:32 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mailout.stack.nl (mailout05.stack.nl [IPv6:2001:610:1108:5010::202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailout.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 546F711B3 for ; Sun, 11 Dec 2016 22:23:32 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mailout.stack.nl (Postfix) with ESMTP id F098D41F5; Sun, 11 Dec 2016 23:23:28 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id DFA3D28494; Sun, 11 Dec 2016 23:23:28 +0100 (CET) Date: Sun, 11 Dec 2016 23:23:28 +0100 From: Jilles Tjoelker To: Konstantin Belousov Cc: FreeBSD Hackers , Jonathan de Boyne Pollard , supervision@list.skarnet.org Subject: Re: NOTE_TRACK, EVFILT_PROC, kqueue, and subreapers Message-ID: <20161211222328.GC17315@stack.nl> References: <20161201174837.GR3428@protected.rcdrun.com> <20161201125438.15230317@mydesk.domain.cxm> <20161206104020.6b2ebb30@eto-mona.office.smartweb.sk> <20161206102637.1ddd152a@mydesk.domain.cxm> <20161207155638.4b2dd629@eto-mona.office.smartweb.sk> <630ace89-e29b-d0d3-9f15-110d8dc3de08@NTLWorld.com> <20161208132842.5d7940bd@eto-mona.office.smartweb.sk> <20161211124637.GA17315@stack.nl> <20161211175451.GV54029@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161211175451.GV54029@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 22:23:32 -0000 On Sun, Dec 11, 2016 at 07:54:51PM +0200, Konstantin Belousov wrote: > On Sun, Dec 11, 2016 at 01:46:38PM +0100, Jilles Tjoelker wrote: > > As reported in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=213928 > > reapers do not receive SIGCHLD when inheriting zombies, although they do > > receive SIGCHLD when an inherited descendant process later terminates > > and are awakened when in a wait call for a matching process. This should > > probably be fixed. > I agree, in principle. Could you update your tests/sys/kern/reaper.c to > include this scenario ? I tested with the patch below. The new test case reaper_sigchld_child_first already passes and I have marked reaper_sigchld_grandchild_first as an expected timeout. > Untested change to the kernel side is below. > diff --git a/sys/kern/kern_exit.c b/sys/kern/kern_exit.c > index f4f453c3556..92899740eef 100644 > --- a/sys/kern/kern_exit.c > +++ b/sys/kern/kern_exit.c > @@ -455,6 +455,12 @@ exit1(struct thread *td, int rval, int signo) > > if (!(q->p_flag & P_TRACED)) { > proc_reparent(q, q->p_reaper); > + if (q->p_state == PRS_ZOMBIE) { > + PROC_LOCK(q->p_reaper); > + pksignal(q->p_reaper, SIGCHLD, q->p_ksi); > + wakeup(q->p_reaper); > + PROC_UNLOCK(q->p_reaper); > + } > } else { > /* > * Traced processes are killed since their existence This change makes reaper_sigchld_grandchild_first pass. The wakeup added here is redundant with the one a few lines above: if (q != NULL) /* only need this if any child is S_ZOMB */ wakeup(q->p_reaper); This is also clear because reaper_wait_grandchild_first already passes. Index: tests/sys/kern/reaper.c =================================================================== --- tests/sys/kern/reaper.c (revision 309839) +++ tests/sys/kern/reaper.c (working copy) @@ -35,6 +35,11 @@ #include #include +static void +dummy_sighandler(int sig __unused, siginfo_t *info __unused, void *ctx __unused) +{ +} + ATF_TC_WITHOUT_HEAD(reaper_wait_child_first); ATF_TC_BODY(reaper_wait_child_first, tc) { @@ -129,6 +134,163 @@ ATF_CHECK_EQ(2, r); } +ATF_TC(reaper_sigchld_child_first); +ATF_TC_HEAD(reaper_sigchld_child_first, tc) +{ + atf_tc_set_md_var(tc, "timeout", "2"); +} +ATF_TC_BODY(reaper_sigchld_child_first, tc) +{ + struct sigaction act; + sigset_t mask; + siginfo_t info; + pid_t parent, child, grandchild, pid; + int r; + int pip[2]; + + /* Be paranoid. */ + pid = waitpid(-1, NULL, WNOHANG); + ATF_REQUIRE(pid == -1 && errno == ECHILD); + + act.sa_sigaction = dummy_sighandler; + act.sa_flags = SA_SIGINFO | SA_RESTART; + r = sigemptyset(&act.sa_mask); + ATF_REQUIRE_EQ(0, r); + r = sigaction(SIGCHLD, &act, NULL); + ATF_REQUIRE_EQ(0, r); + + r = sigemptyset(&mask); + ATF_REQUIRE_EQ(0, r); + r = sigaddset(&mask, SIGCHLD); + ATF_REQUIRE_EQ(0, r); + r = sigprocmask(SIG_BLOCK, &mask, NULL); + ATF_REQUIRE_EQ(0, r); + + parent = getpid(); + r = procctl(P_PID, parent, PROC_REAP_ACQUIRE, NULL); + ATF_REQUIRE_EQ(0, r); + + r = pipe(pip); + ATF_REQUIRE_EQ(0, r); + + child = fork(); + ATF_REQUIRE(child != -1); + if (child == 0) { + if (close(pip[1]) != 0) + _exit(100); + grandchild = fork(); + if (grandchild == -1) + _exit(101); + else if (grandchild == 0) { + if (read(pip[0], &(uint8_t){ 0 }, 1) != 0) + _exit(102); + if (getppid() != parent) + _exit(103); + _exit(2); + } else + _exit(3); + } + + r = sigwaitinfo(&mask, &info); + ATF_REQUIRE_EQ(SIGCHLD, r); + ATF_CHECK_EQ(SIGCHLD, info.si_signo); + ATF_CHECK_EQ(CLD_EXITED, info.si_code); + ATF_CHECK_EQ(3, info.si_status); + ATF_CHECK_EQ(child, info.si_pid); + + pid = waitpid(child, NULL, 0); + ATF_REQUIRE_EQ(child, pid); + + r = close(pip[1]); + ATF_REQUIRE_EQ(0, r); + + r = sigwaitinfo(&mask, &info); + ATF_REQUIRE_EQ(SIGCHLD, r); + ATF_CHECK_EQ(SIGCHLD, info.si_signo); + ATF_CHECK_EQ(CLD_EXITED, info.si_code); + ATF_CHECK_EQ(2, info.si_status); + grandchild = info.si_pid; + ATF_REQUIRE(grandchild > 0); + ATF_REQUIRE(grandchild != parent); + ATF_REQUIRE(grandchild != child); + + pid = waitpid(-1, NULL, 0); + ATF_REQUIRE_EQ(grandchild, pid); + + r = close(pip[0]); + ATF_REQUIRE_EQ(0, r); +} + +ATF_TC(reaper_sigchld_grandchild_first); +ATF_TC_HEAD(reaper_sigchld_grandchild_first, tc) +{ + atf_tc_set_md_var(tc, "timeout", "2"); +} +ATF_TC_BODY(reaper_sigchld_grandchild_first, tc) +{ + struct sigaction act; + sigset_t mask; + siginfo_t info; + pid_t parent, child, grandchild, pid; + int r; + + atf_tc_expect_timeout("bug 213928"); + + /* Be paranoid. */ + pid = waitpid(-1, NULL, WNOHANG); + ATF_REQUIRE(pid == -1 && errno == ECHILD); + + act.sa_sigaction = dummy_sighandler; + act.sa_flags = SA_SIGINFO | SA_RESTART; + r = sigemptyset(&act.sa_mask); + ATF_REQUIRE_EQ(0, r); + r = sigaction(SIGCHLD, &act, NULL); + ATF_REQUIRE_EQ(0, r); + + r = sigemptyset(&mask); + ATF_REQUIRE_EQ(0, r); + r = sigaddset(&mask, SIGCHLD); + ATF_REQUIRE_EQ(0, r); + r = sigprocmask(SIG_BLOCK, &mask, NULL); + ATF_REQUIRE_EQ(0, r); + + parent = getpid(); + r = procctl(P_PID, parent, PROC_REAP_ACQUIRE, NULL); + ATF_REQUIRE_EQ(0, r); + + child = fork(); + ATF_REQUIRE(child != -1); + if (child == 0) { + grandchild = fork(); + if (grandchild == -1) + _exit(101); + else if (grandchild == 0) + _exit(2); + else { + if (waitid(P_PID, grandchild, NULL, + WNOWAIT | WEXITED) != 0) + _exit(102); + _exit(3); + } + } + + pid = waitpid(child, NULL, 0); + ATF_REQUIRE_EQ(child, pid); + + r = sigwaitinfo(&mask, &info); + ATF_REQUIRE_EQ(SIGCHLD, r); + ATF_CHECK_EQ(SIGCHLD, info.si_signo); + ATF_CHECK_EQ(CLD_EXITED, info.si_code); + ATF_CHECK_EQ(2, info.si_status); + grandchild = info.si_pid; + ATF_REQUIRE(grandchild > 0); + ATF_REQUIRE(grandchild != parent); + ATF_REQUIRE(grandchild != child); + + pid = waitpid(-1, NULL, 0); + ATF_REQUIRE_EQ(grandchild, pid); +} + ATF_TC_WITHOUT_HEAD(reaper_status); ATF_TC_BODY(reaper_status, tc) { @@ -484,6 +646,8 @@ ATF_TP_ADD_TC(tp, reaper_wait_child_first); ATF_TP_ADD_TC(tp, reaper_wait_grandchild_first); + ATF_TP_ADD_TC(tp, reaper_sigchld_child_first); + ATF_TP_ADD_TC(tp, reaper_sigchld_grandchild_first); ATF_TP_ADD_TC(tp, reaper_status); ATF_TP_ADD_TC(tp, reaper_getpids); ATF_TP_ADD_TC(tp, reaper_kill_badsig); -- Jilles Tjoelker From owner-freebsd-hackers@freebsd.org Mon Dec 12 04:21:19 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5DC60C73EEE; Mon, 12 Dec 2016 04:21:19 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-1.mit.edu (dmz-mailsec-scanner-1.mit.edu [18.9.25.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ED1F9B1A; Mon, 12 Dec 2016 04:21:18 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 1209190c-2ebff70000004ea7-c1-584e25b465f1 Received: from mailhub-auth-3.mit.edu ( [18.9.21.43]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id 9B.DE.20135.4B52E485; Sun, 11 Dec 2016 23:21:09 -0500 (EST) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id uBC4L8sk023164; Sun, 11 Dec 2016 23:21:08 -0500 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id uBC4L4oM029556 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 11 Dec 2016 23:21:07 -0500 Date: Sun, 11 Dec 2016 22:21:04 -0600 From: Benjamin Kaduk To: freebsd-hackers@FreeBSD.org Cc: freebsd-current@FreeBSD.org Subject: Call for 2016Q4 quarterly status reports Message-ID: <20161212042104.GQ8460@kduck.kaduk.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.6.1 (2016-04-27) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrLIsWRmVeSWpSXmKPExsUixCmqrbtV1S/C4PZybYs5bz4wWWzf/I/R gcljxqf5LAGMUVw2Kak5mWWpRfp2CVwZd1ZMZS6Yw13xa8E/1gbGNZxdjJwcEgImEm9mLWUE sYUE2pgkuhf5dzFyAdkbGSW+7bjJCOFcZZK43LOVHaSKRUBV4vfJc2A2m4CaxPoV15hBbBEB eYl9Te/B4sxA9q+tTWC2sIChxPeOLWAbeAWMJZYuXsoOYQtKnJz5hAWiXkvixr+XTF2MHEC2 tMTyfxwgYVEBZYmGGQ+YJzDyzULSMQtJxyyEjgWMzKsYZVNyq3RzEzNzilOTdYuTE/PyUot0 DfVyM0v0UlNKNzGCQ02SZwfjmTdehxgFOBiVeHhnpPtGCLEmlhVX5h5ilORgUhLlZVwHFOJL yk+pzEgszogvKs1JLT7EKMHBrCTCy6viFyHEm5JYWZValA+TkuZgURLnvZTpHiEkkJ5Ykpqd mlqQWgSTleHgUJLgjQVpFCxKTU+tSMvMKUFIM3FwggznARq+XBlkeHFBYm5xZjpE/hSjopQ4 73yQhABIIqM0D64XlAoksvfXvGIUB3pFmLccZAUPMI3Adb8CGswENPj5Pm+QwSWJCCmpBsZz RpPiTlsuyJtWn3Ph5r9137kLrRUFJY+sVLw+9e3bWZ9uX3OL9NZZYNSV/dK0oewRy3fhKXuX Fd7M3jXnye9bVY1fxQPjXy87cj9p18LXvfvkLUzVzonwR5run5SuvYBn5xGXuqWuUoqbQ6Rm RIjEyYrNz40wajiz0Eo8bqXal37P4EKRj4xKLMUZiYZazEXFiQAJICN94AIAAA== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 04:21:19 -0000 Dear FreeBSD Community, The deadline for the next FreeBSD Quarterly Status update is January 7, 2016, for work done in October through December. Status report submissions do not need to be very long. They may be about anything happening in the FreeBSD project and community, and provide a great way to inform FreeBSD users and developers about work that is underway and completed. Submission of reports is not restricted to committers; anyone doing anything interesting and FreeBSD related can -- and should -- write one! The preferred and easiest submission method is to use the XML generator [1] with the results emailed to the status report team at monthly@FreeBSD.org . (Do be sure, though, to save the form output and not the form itself!) There is also an XML template [2] that can be filled out manually and attached if preferred. For the expected content and style, please study our guidelines on how to write a good status report [3]. You can also review previous issues [4][5] for ideas on the style and format. We look forward to seeing your 2016Q4 reports! Thanks, Ben (on behalf of monthly@) [1] https://www.FreeBSD.org/cgi/monthly.cgi [2] https://www.FreeBSD.org/news/status/report-sample.xml [3] https://www.FreeBSD.org/news/status/howto.html [4] https://www.FreeBSD.org/news/status/report-2016-04-2016-06.html [5] https://www.FreeBSD.org/news/status/report-2016-07-2016-09.html From owner-freebsd-hackers@freebsd.org Mon Dec 12 10:45:16 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13783C7173A for ; Mon, 12 Dec 2016 10:45:16 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (vps.rulingia.com [103.243.244.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.rulingia.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 97573AE3; Mon, 12 Dec 2016 10:45:15 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (ppp59-167-167-3.static.internode.on.net [59.167.167.3]) by vps.rulingia.com (8.15.2/8.15.2) with ESMTPS id uBCAZofr075219 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 12 Dec 2016 21:35:56 +1100 (AEDT) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.15.2/8.15.2) with ESMTPS id uBCAZhc1004298 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 12 Dec 2016 21:35:43 +1100 (AEDT) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.15.2/8.15.2/Submit) id uBCAZg7D004297; Mon, 12 Dec 2016 21:35:42 +1100 (AEDT) (envelope-from peter) Date: Mon, 12 Dec 2016 21:35:42 +1100 From: Peter Jeremy To: Jonathan Anderson Cc: freebsd-hackers@freebsd.org Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems Message-ID: <20161212103542.GN61036@server.rulingia.com> References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="U+BazGySraz5kW0T" Content-Disposition: inline In-Reply-To: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.7.1 (2016-10-04) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 10:45:16 -0000 --U+BazGySraz5kW0T Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2016-Dec-11 16:42:38 -0330, Jonathan Anderson wro= te: >I, for one, agree, and at the risk of being slightly provocative, I=20 >think it could be taken even further. :) I'd love to be able to retrieve= =20 >a sysctl entry or subtree as an nvlist_t. Rather than doing lots of=20 >system calls to enumerate clearly-related sysctl values, it would be=20 >very cool to do one system call and then inspect the resulting data=20 >structure in memory. We already use libnv in a number of places, and it=20 >seems like it could be a natural mapping from the sysctl hierarchy. This is just pushing the work from userland to the kernel. There's no sysctl tree in the kernel that can be readily exported as a nvlist - the kernel would need to iterate through the hierarchy and build the tree. The only advantage of this approach is that a client only needs to perform a single syscall. IMO, this is the wrong approach - where practical, functionality should be in userland, not in the kernel. --=20 Peter Jeremy --U+BazGySraz5kW0T Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJYTn1+XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFRUIyOTg2QzMwNjcxRTc0RTY1QzIyN0Ux NkE1OTdBMEU0QTIwQjM0AAoJEBall6Dkogs0P18P/ixlj+0H68Og5xbc7F7ARohd +UNN+OPg3UJxQc9iHzc8IOkvbuaarWPN2RB4zTDl3Pr5pOjTCGkhJEV8shXyhQHS WhMXG+DXuXlShc1oMoJvZcQNZVM2N4G8idIMfWfCdeNT/cTq45J9A9/T9SRntcoD D6zy/1cChyo7soZlxIRKgSNt3xLzgw3jnvIYt39REqhod2p3Z0P239PbT1eO0VSg e99mB8PcSRQBy9BsRX+Ci6mD9WCqdQ/TY7ReXVed3ZIWp49qSUPfBcVUKniSf+5M HHja4NSAwE/VpuS4Iq0ABDKVejHanUDOuF3VudjOV7XiZ+vCb7v4tQP2UCYPANyF NKNoECETia9AjvDE7sJcvcUWLy/DVUvyY91uMkPofGD4CX6Dkxzd4EvmcfhrlZPn 9CxHHWDmMrKH2K4M7sPEZb1msdrKlqLkIYRvvmRkwySCjGHpwyIi5IlMSeKF4+3m WNItK48Ug4VsTGkcEp9C4uydws4BwqG7Tf5Mm2oQBxnW+bHBAhFGJpcO+weOWD7t Dm4M3TsoilvVpnc7k8Gj+GweDbVOIk/xt3uoBVjfeTnLcjSUVvHaD58KiSsKtMJY xaX+1JNfadD2qxblihlLOdyLxwMZfvbe7tUHk/bYS3T6VYj4RZq9Qvacq1iPzDLo xu1qFlp8ZbMiJTAxIa5d =Qw/4 -----END PGP SIGNATURE----- --U+BazGySraz5kW0T-- From owner-freebsd-hackers@freebsd.org Mon Dec 12 11:12:25 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C464C723A8 for ; Mon, 12 Dec 2016 11:12:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C7B231FB9 for ; Mon, 12 Dec 2016 11:12:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBCBCIJR089927 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 12 Dec 2016 13:12:18 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBCBCIJR089927 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBCBCIqk089926; Mon, 12 Dec 2016 13:12:18 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 12 Dec 2016 13:12:18 +0200 From: Konstantin Belousov To: Jilles Tjoelker Cc: FreeBSD Hackers , Jonathan de Boyne Pollard , supervision@list.skarnet.org Subject: Re: NOTE_TRACK, EVFILT_PROC, kqueue, and subreapers Message-ID: <20161212111218.GB54029@kib.kiev.ua> References: <20161201125438.15230317@mydesk.domain.cxm> <20161206104020.6b2ebb30@eto-mona.office.smartweb.sk> <20161206102637.1ddd152a@mydesk.domain.cxm> <20161207155638.4b2dd629@eto-mona.office.smartweb.sk> <630ace89-e29b-d0d3-9f15-110d8dc3de08@NTLWorld.com> <20161208132842.5d7940bd@eto-mona.office.smartweb.sk> <20161211124637.GA17315@stack.nl> <20161211175451.GV54029@kib.kiev.ua> <20161211222328.GC17315@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161211222328.GC17315@stack.nl> User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 11:12:25 -0000 On Sun, Dec 11, 2016 at 11:23:28PM +0100, Jilles Tjoelker wrote: > On Sun, Dec 11, 2016 at 07:54:51PM +0200, Konstantin Belousov wrote: > > On Sun, Dec 11, 2016 at 01:46:38PM +0100, Jilles Tjoelker wrote: > > > As reported in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=213928 > > > reapers do not receive SIGCHLD when inheriting zombies, although they do > > > receive SIGCHLD when an inherited descendant process later terminates > > > and are awakened when in a wait call for a matching process. This should > > > probably be fixed. > > I agree, in principle. Could you update your tests/sys/kern/reaper.c to > > include this scenario ? > > I tested with the patch below. The new test case > reaper_sigchld_child_first already passes and I have marked > reaper_sigchld_grandchild_first as an expected timeout. > > > Untested change to the kernel side is below. > > > diff --git a/sys/kern/kern_exit.c b/sys/kern/kern_exit.c > > index f4f453c3556..92899740eef 100644 > > --- a/sys/kern/kern_exit.c > > +++ b/sys/kern/kern_exit.c > > @@ -455,6 +455,12 @@ exit1(struct thread *td, int rval, int signo) > > > > if (!(q->p_flag & P_TRACED)) { > > proc_reparent(q, q->p_reaper); > > + if (q->p_state == PRS_ZOMBIE) { > > + PROC_LOCK(q->p_reaper); > > + pksignal(q->p_reaper, SIGCHLD, q->p_ksi); > > + wakeup(q->p_reaper); > > + PROC_UNLOCK(q->p_reaper); > > + } > > } else { > > /* > > * Traced processes are killed since their existence > > This change makes reaper_sigchld_grandchild_first pass. > > The wakeup added here is redundant with the one a few lines above: > if (q != NULL) /* only need this if any child is S_ZOMB */ > wakeup(q->p_reaper); I removed additional wakeup, it should be innocent. Might be the better change is to remove the lines you cited above and do wakeups inside the loop after reparenting. But I kept it that way for now. > > This is also clear because reaper_wait_grandchild_first already passes. > > Index: tests/sys/kern/reaper.c Thank you, I verified that all tests pass with the patched kernel and removed timeouts from your reaper_sigchld_*child_first new tests. The kernel chunk is committed as r309886, please commit tests. From owner-freebsd-hackers@freebsd.org Mon Dec 12 13:24:01 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A12EC7378A for ; Mon, 12 Dec 2016 13:24:01 +0000 (UTC) (envelope-from hrant@dadivanyan.net) Received: from pandora.amnic.net (pandora.amnic.net [IPv6:2001:67c:21c:a610::11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CECCBE70 for ; Mon, 12 Dec 2016 13:24:00 +0000 (UTC) (envelope-from hrant@dadivanyan.net) Received: from ran by pandora.amnic.net with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1cGQaC-0005ZI-Mh for freebsd-hackers@freebsd.org; Mon, 12 Dec 2016 17:23:56 +0400 Subject: system time instability To: freebsd-hackers@freebsd.org Date: Mon, 12 Dec 2016 17:23:56 +0400 (AMT) From: Hrant Dadivanyan X-PGP: https://amnic.net/pgpkeys/hrant.asc X-NCC-RegID: am.isoc X-Mailer: ELM [version 2.4ME+ PL127 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="UTF-8" Message-Id: X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 13:24:01 -0000 Hello, After upgrade of stratum 1 ntp server hardware from a Via EPIA Mini-ITX to Supermicro PDSBM-LN2 and OS from 8.4/i386 to 10.3-RELEASE-p12/amd64 it starts to work unstable. Most of the time it keeps time pretty well with offset less than 1-2 us, but once a few hours pll frequency jumps, then clock drifts. After passing calibration interval time (256s) frequency returns back to normal, then, after appropriate time, clock stabilizes again. Excerpt from loopstats: 57734 37624.525 -0.000000955 0.120 0.000000588 0.000211 4 57734 37640.526 0.000000319 0.120 0.000000506 0.000198 4 57734 37656.526 -0.000000789 0.120 0.001081214 0.000185 4 57734 37672.526 -0.000398921 100.120 0.000154630 35.355339 4 57734 37688.526 -0.001941140 100.120 0.000188374 33.071891 4 57734 37704.525 -0.003389196 100.120 0.000177488 30.935922 4 57734 37720.525 -0.004745689 100.120 0.000166147 28.937905 4 57734 37736.525 -0.006022007 100.120 0.000156269 27.068931 4 57734 37752.526 -0.007220430 100.120 0.000146663 25.320667 4 57734 37768.526 -0.008343331 100.120 0.000137805 23.685315 4 57734 37784.525 -0.009399651 100.120 0.000129406 22.155583 4 57734 37800.525 -0.010391390 100.120 0.000121937 20.724651 4 57734 37816.526 -0.011320293 100.120 0.000114053 19.386136 4 57734 37832.526 -0.012194902 100.120 0.000107191 18.134069 4 57734 37848.526 -0.013013037 100.120 0.000100035 16.962869 4 57734 37864.526 -0.013783932 100.120 0.000094497 15.867311 4 57734 37880.526 -0.014507271 100.120 0.000088691 14.842510 4 57734 37896.525 -0.015184384 100.120 0.000083266 13.883897 4 57734 37912.526 -0.015822296 100.120 0.000078249 12.987196 4 57734 37928.525 -0.016119704 0.122 0.000103405 37.383615 4 57734 37944.526 -0.015132723 0.122 0.000120509 34.969170 4 57734 37960.526 -0.014207941 0.122 0.000113355 32.710663 4 57734 37976.525 -0.013339661 0.122 0.000107051 30.598023 4 [snip] 57734 40296.525 -0.000000337 0.122 0.000001621 0.002136 4 57734 40312.526 -0.000000980 0.122 0.000001635 0.001998 4 The change in pll frequency is usually 100ppm, but not always. For today, for example, it's 29ppm once, 69.3ppm once and 100ppm three times. Had tried three available timecounters: TSC-low, ACPI-fast, HPET. Had changed eventtimer from HPET to LAPIC, kern.eventtimer.periodic from 0 to 1. All the changes are followed by service ntpd restart. Also tried to change kern.hz from 1000 to 100. Had even tried 11.0 on other, but the exactly same board. The original board has OCXO instead of quartz, but reconnecting the original quartz doesn't help. Didn't try another hardware and/or OS yet, the server isn't easy reachable, but, in lack of better ideas, will definitely try. Kernel has stripped all unused drivers and options plus PPS_SYNC, then FFCLOCK added. All the additions: options IPSEC options GEOM_ELI options PPS_SYNC options FFCLOCK device crypto device enc device pf device pflog device smbus device ichsmb device smb device coretemp device cpuctl device nvram device smbios device ipmi device aesni The relevant part of ntp.conf: fudge 127.127.20.0 time2 0.6 flag1 1 flag2 0 flag3 1 server 127.127.20.0 mode 2 minpoll 4 prefer server minpoll 8 iburst restrict default limited kod nomodify notrap nopeer noquery rc.conf: ntpd_program="/usr/local/sbin/ntpd" ntpd_config="/etc/ntpd.conf" ntpd_flags="-N -p /var/run/ntpd.pid -f /var/db/ntpd.drift" ntpd_sync_on_start="YES" sysctl.conf (this change is also seems irrelevant, rebooting without this frequency correction changes nothing in the behaviour): machdep.tsc_freq=2194498500 # pll freq offset change from 21.678 to 0.120ppm $ sysctl kern.hz kern.timecounter kern.eventtimer kern.hz: 1000 kern.timecounter.tsc_shift: 1 kern.timecounter.smp_tsc_adjust: 0 kern.timecounter.smp_tsc: 1 kern.timecounter.invariant_tsc: 1 kern.timecounter.fast_gettime: 1 kern.timecounter.tick: 1 kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) i8254(0) HPET(950) dummy(-1000000) kern.timecounter.hardware: TSC-low kern.timecounter.alloweddeviation: 5 kern.timecounter.stepwarnings: 0 kern.timecounter.tc.TSC-low.quality: 1000 kern.timecounter.tc.TSC-low.frequency: 1097249250 kern.timecounter.tc.TSC-low.counter: 2359573202 kern.timecounter.tc.TSC-low.mask: 4294967295 kern.timecounter.tc.ACPI-fast.quality: 900 kern.timecounter.tc.ACPI-fast.frequency: 3579545 kern.timecounter.tc.ACPI-fast.counter: 9238615 kern.timecounter.tc.ACPI-fast.mask: 16777215 kern.timecounter.tc.i8254.quality: 0 kern.timecounter.tc.i8254.frequency: 1193182 kern.timecounter.tc.i8254.counter: 9906 kern.timecounter.tc.i8254.mask: 65535 kern.timecounter.tc.HPET.quality: 950 kern.timecounter.tc.HPET.frequency: 14318180 kern.timecounter.tc.HPET.counter: 2305610093 kern.timecounter.tc.HPET.mask: 4294967295 kern.eventtimer.periodic: 0 kern.eventtimer.timer: HPET kern.eventtimer.idletick: 0 kern.eventtimer.singlemul: 2 kern.eventtimer.choice: HPET(450) HPET1(440) HPET2(440) LAPIC(400) i8254(100) RTC(0) kern.eventtimer.et.i8254.quality: 100 kern.eventtimer.et.i8254.frequency: 1193182 kern.eventtimer.et.i8254.flags: 1 kern.eventtimer.et.RTC.quality: 0 kern.eventtimer.et.RTC.frequency: 32768 kern.eventtimer.et.RTC.flags: 17 kern.eventtimer.et.HPET2.quality: 440 kern.eventtimer.et.HPET2.frequency: 14318180 kern.eventtimer.et.HPET2.flags: 3 kern.eventtimer.et.HPET1.quality: 440 kern.eventtimer.et.HPET1.frequency: 14318180 kern.eventtimer.et.HPET1.flags: 3 kern.eventtimer.et.HPET.quality: 450 kern.eventtimer.et.HPET.frequency: 14318180 kern.eventtimer.et.HPET.flags: 3 kern.eventtimer.et.LAPIC.quality: 400 kern.eventtimer.et.LAPIC.frequency: 99749970 kern.eventtimer.et.LAPIC.flags: 15 $ Any hints ? Thank you, Hrant -- Hrant Dadivanyan (aka Ran d'Adi) hrant(at)dadivanyan.net /* "Feci quod potui, faciant meliora potentes." */ ran(at)psg.com From owner-freebsd-hackers@freebsd.org Mon Dec 12 13:35:06 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 118F6C73AF1 for ; Mon, 12 Dec 2016 13:35:06 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: from mail-yw0-x235.google.com (mail-yw0-x235.google.com [IPv6:2607:f8b0:4002:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C51881342 for ; Mon, 12 Dec 2016 13:35:05 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: by mail-yw0-x235.google.com with SMTP id r204so64544995ywb.0 for ; Mon, 12 Dec 2016 05:35:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuxi-nl.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=FM+hi0Rfh3vkazbXXh6sbkCUArAtalccChSbN7n4DDg=; b=M7FRGaj1iNRg3AS+5OOFSZRW+78MkCYGD25boHtwhWK+BAtb4Unb4v1zAlEBkZfJc3 lCMuqR84GMskiW+GjHDenS+XV8fuJwvUFZNzUPgXucQhORAjj5GpDXnjsr3i0xI8AVxX wvUg/TJydhMEqJurmZS7up0Ddy5hLoVlEO88Gz2zWCzTwJLKVLo2mpJM0E6DQyFq8MYV dlKsd+xpMdZLNQ33pFsvCgDpLMfiOsti4NViEZA67pUmkrvpWxHtlaf8eplTCdTCUh3k 9ANCNk7sUMzAR9XH+MzzC2fby/A5smZhX58aLoOki6K84GKL7y8IZiHKnO8QWWp0oRhI HYkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=FM+hi0Rfh3vkazbXXh6sbkCUArAtalccChSbN7n4DDg=; b=PHivJmgVAZyaFR9jxWRlUYfOqYbCobQFRjpcwI6xqewPZgG1a+LrOwazTXHpdAD9vi o4lV9DadD5+ZaevcXBglN1jRs7s5Q1Kgueldxg98md6Xp3/zzsPR+WRbIz2fItLCfqkM 6EP3Vpd2+W1iRPtKxl7hfrLbEyFF9aRmuUnr6SepjApLTySqruH3wVim6B3Jf1agrZ45 MBo1c7hncYt7wWxkTfIKcMa30JcEe/v2ME+ucXjfgSXidVBPVNWbH9a3BhmXWSy+TIYB jGj/lJVfaElsgvTe71oEipXU1pzKD/ZnLFyazsdmvnpmw8DpLSmTfd319ZfatNneFbrO hnUQ== X-Gm-Message-State: AKaTC00pWatxaVhBmtxA1YbhRsf3z/nqVfhvoCETLagKXnL15Jc/B/G6sk6AUQ0Ol09H5Tv/Do29ZH+KH5MEVg== X-Received: by 10.13.246.193 with SMTP id g184mr89165767ywf.277.1481549704984; Mon, 12 Dec 2016 05:35:04 -0800 (PST) MIME-Version: 1.0 Received: by 10.129.0.212 with HTTP; Mon, 12 Dec 2016 05:34:34 -0800 (PST) In-Reply-To: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> From: Ed Schouten Date: Mon, 12 Dec 2016 14:34:34 +0100 Message-ID: Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: Jonathan Anderson Cc: Allan Jude , FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 13:35:06 -0000 Hi Jonathan, 2016-12-11 21:12 GMT+01:00 Jonathan Anderson : > Rather than doing lots of system calls to > enumerate clearly-related sysctl values, it would be very cool to do one > system call and then inspect the resulting data structure in memory. I'm not sure it's worth tackling this problem at this time. My current version of the exporter is already pretty fast: $ time ./prometheus_sysctl_exporter | wc -l 0.00 real 0.00 user 0.00 sys 3162 If our sysctl tree would get substantially larger at some point, we should of course revisit this. -- Ed Schouten Nuxi, 's-Hertogenbosch, the Netherlands KvK-nr.: 62051717 From owner-freebsd-hackers@freebsd.org Mon Dec 12 14:16:30 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38823C7284C for ; Mon, 12 Dec 2016 14:16:30 +0000 (UTC) (envelope-from larry.maloney@hackerdojo.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 16D1795B for ; Mon, 12 Dec 2016 14:16:30 +0000 (UTC) (envelope-from larry.maloney@hackerdojo.com) Received: by mailman.ysv.freebsd.org (Postfix) id 1618EC7284B; Mon, 12 Dec 2016 14:16:30 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 15B57C7284A for ; Mon, 12 Dec 2016 14:16:30 +0000 (UTC) (envelope-from larry.maloney@hackerdojo.com) Received: from mail-pg0-x242.google.com (mail-pg0-x242.google.com [IPv6:2607:f8b0:400e:c05::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E712E95A for ; Mon, 12 Dec 2016 14:16:29 +0000 (UTC) (envelope-from larry.maloney@hackerdojo.com) Received: by mail-pg0-x242.google.com with SMTP id p66so1111830pga.2 for ; Mon, 12 Dec 2016 06:16:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hackerdojo.com; s=google; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=fR7QSC+wcvE1IMgSR4uEvKjkVjLQleoNwIC04hzaBv8=; b=kL1k9aT74tphbvg0abs34B/vA1xytlmTkiHTM747ifCh1usQGz9zqx0NFBlfUERNqc JsVBnJ8yttcnjlflO0w/HJ/8dj3y8Pr4EwDbP46f8atapT39B1xHczZaWyEP6iQzSgLD LqpxCLI+UHqsxdZaS2WO1Ufi7zc0lALD7zxZI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=fR7QSC+wcvE1IMgSR4uEvKjkVjLQleoNwIC04hzaBv8=; b=SUwf5tRSJuSrA1yblYclE2sJ/k/ftiNUxSXMtoneV0XTZZrBucMRokhRpmBHKlP2lB JGFXySKEk0Kk8H3qXXGBxq5ir+Kn22Y4nRnIKzeU73xQLKSNy/hDLQ6RlMopW2ZqQN0m ReKUcHtimH7ARC2OdpwJavhbp/F8so7xhnlRmhLJlRKF1id+PwVfNX/yUamaaR56bENg htYSph9bPPiDQ7ocEhkd97DkiQJr2eyoYVLXjgVJhHv/Oi+Jv/JO0qCqzmszEl6zyDE9 OwX1xGKSS+W/aQgcYWNShJjDxGLdSEcQ2ILPAAaxhJUzQ4rwQyw8Ia+yyQSWWP0yIm9e b0jA== X-Gm-Message-State: AKaTC03oxsrtrbQKXXkuHT7J/Q/1wzbOl4nCd/g7g0WdntOgxuC7FOz/oN0mzF710WRiNDQA X-Received: by 10.98.76.8 with SMTP id z8mr96341341pfa.167.1481552189338; Mon, 12 Dec 2016 06:16:29 -0800 (PST) Received: from [10.0.0.14] (c-73-202-177-47.hsd1.ca.comcast.net. [73.202.177.47]) by smtp.gmail.com with ESMTPSA id 1sm76878575pgp.1.2016.12.12.06.16.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Dec 2016 06:16:27 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems From: Larry Maloney X-Mailer: iPhone Mail (14B72) In-Reply-To: Date: Mon, 12 Dec 2016 06:16:26 -0800 Cc: hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <7E6EAE02-F059-4C88-8D43-21F0811BC07B@hackerdojo.com> References: To: Ed Schouten X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 14:16:30 -0000 Cool! I like it Sent from my iPhone > On Dec 11, 2016, at 11:35 AM, Ed Schouten wrote: >=20 > Hi there, >=20 > The last couple of months I've been playing around with a monitoring > system called Prometheus (https://prometheus.io/). In short, > Prometheus works like this: >=20 > ----- If you already know Prometheus, skip this ----- >=20 > 1. For the thing you want to monitor, you either integrate the > Prometheus client library into your codebase or you write a separate > exporter process. The client library or the exporter process then > exposes key metrics of your application over HTTP. Simplified example: >=20 > $ curl http://localhost:12345/metrics > # HELP open_file_descriptors The number of files opened by the process > # TYPE open_file_descriptors gauge > open_file_descriptors 12 > # HELP http_requests The number of HTTP requests received. > # TYPE http_requests counter > http_requests{result=3D"2xx"} 100 > http_requests{result=3D"4xx"} 14 > http_requests{result=3D"5xx"} 0 >=20 > 2. You fire op Prometheus and configure it to scrape and store all of > the things you want to monitor. Prometheus can then add more labels to > the metrics it scrapes. So the example above may get transformed by > Prometheus to look like this: >=20 > open_file_descriptors{job=3D"nginx",instance=3D"web1.mycompany.com"} 12 > http_requests{job=3D"nginx",instance=3D"web1.mycompany.com",result=3D"2xx"= } 100 > http_requests{job=3D"nginx",instance=3D"web1.mycompany.com",result=3D"4xx"= } 14 > http_requests{job=3D"nginx",instance=3D"web1.mycompany.com",result=3D"5xx"= } 0 >=20 > Fun fact: Prometheus can also scrape Prometheus, so if you operate > multiple datacenters, you can let a global instance scrape a per-DC > instance and add a dc=3D"..." label to all metrics. >=20 > 3. After scraping data for some time, you can do fancy queries like these:= >=20 > - Compute the 5-minute rate of HTTP requests per server and per HTTP error= code: > rate(http_requests[5m]) >=20 > - Compute the 5-minute rate of all HTTP requests on the entire cluster: > sum(rate(http_requests[5m])) >=20 > - Same as the above, but aggregate by HTTP error code: > sum(rate(http_requests[5m])) by (result) >=20 > Prometheus can do alerting as well by using these expressions as matchers.= >=20 > 4. Set up Grafana and voila: you can create fancy dashboards! >=20 > ----- If you skipped the introduction, start reading here ----- >=20 > The Prometheus folks have developed a tool called the node_exporter > (https://github.com/prometheus/node_exporter). Basically it extracts a > whole bunch of interesting system-related metrics (disk usage, network > I/O, etc) through sysctl(3), invoking ioctl(2), parsing /proc files, > etc. and exposes that information using Prometheus' syntax. >=20 > The other day I was thinking: in a certain way, the node exporter is a > bit of a redundant tool on the BSDs. Instead of needing to write > custom collectors for every kernel subsystem, we could write a generic > exporter for converting the entire sysctl(3) tree to Prometheus > metrics, which is exactly what I'm experimenting with here: >=20 > https://github.com/EdSchouten/prometheus_sysctl_exporter >=20 > An example of what this tool's output looks like: >=20 > $ ./prometheus_sysctl_exporter > ... > # HELP kern_maxfiles Maximum number of files > sysctl_kern_maxfiles 1043382 > # HELP kern_openfiles System-wide number of open files > sysctl_kern_openfiles 316 > ... >=20 > You could use this to write alerting rules like this: >=20 > ALERT FileDescriptorUsageHigh > IF sysctl_kern_openfiles / sysctl_kern_maxfiles > 0.5 > FOR 15m > ANNOTATIONS { > description =3D "More than half of all FDs are in use!", > } >=20 > There you go. Access to a very large number of metrics without too much ef= fort. >=20 > My main question here is: are there any people in here interested in > seeing something like this being developed into something usable? If > so, let me know and I'll pursue this further. >=20 > I also have a couple of technical questions related to sysctl(3)'s > in-kernel design: >=20 > - Prometheus differentiates between gauges (memory usage), counters > (number of HTTP requests), histograms (per-RPC latency stats), etc., > while sysctl(3) does not. It would be nice if we could have that info > on a per-sysctl basis. Mind if I add a CTLFLAG_GAUGE, CTLFLAG_COUNTER, > etc? >=20 > - Semantically sysctl(3) and Prometheus are slightly different. > Consider this sysctl: >=20 > hw.acpi.thermal.tz0.temperature: 27.8C >=20 > My tool currently converts this metric's name to > sysctl_hw_acpi_thermal_tz0_temperature. This is suboptimal, as it > would ideally be called > sysctl_hw_acpi_thermal_temperature{sensor=3D"tz0"}. Otherwise you > wouldn't be able to write generic alerting rules, use aggregation in > queries, etc. >=20 > I was thinking: we could quite easily do such a translation by > attaching labels to SYSCTL_NODE objects. As in, the hw.acpi.thermal > node would get a label "sensor". Any OID placed underneath this node > will not become a midfix of the sysctl name, but the value of that > label instead. Thoughts? >=20 > A final remark I want to make: a concern might be that changes like > these would not be generic, but only apply to Prometheus. I tend to > disagree. First of all, an advantage of Prometheus is that the > coupling is very loose: it's just a GET request with key-value pairs. > Anyone is free to add his/her own implementation. >=20 > Second, emaste@ also pointed me to another monitoring framework being > developed by Intel right now: >=20 > https://github.com/intelsdi-x/snap >=20 > The changes I'm proposing would also seem to make exporting sysctl > data to that system easier. >=20 > Anyway, thanks for reading this huge wall of text. >=20 > Best regards, > --=20 > Ed Schouten > Nuxi, 's-Hertogenbosch, the Netherlands > KvK-nr.: 62051717 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"= From owner-freebsd-hackers@freebsd.org Mon Dec 12 15:30:51 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F140C731D3 for ; Mon, 12 Dec 2016 15:30:51 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 51C581579 for ; Mon, 12 Dec 2016 15:30:50 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: d4e39c35-c07f-11e6-9f67-d3961ed5a660 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id d4e39c35-c07f-11e6-9f67-d3961ed5a660; Mon, 12 Dec 2016 15:29:55 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id uBCFTfBJ008513; Mon, 12 Dec 2016 08:29:41 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1481556581.1889.322.camel@freebsd.org> Subject: Re: system time instability From: Ian Lepore To: Hrant Dadivanyan , freebsd-hackers@freebsd.org Date: Mon, 12 Dec 2016 08:29:41 -0700 In-Reply-To: References: Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 15:30:51 -0000 On Mon, 2016-12-12 at 17:23 +0400, Hrant Dadivanyan wrote: > Hello, > > After upgrade of stratum 1 ntp server hardware from a Via EPIA Mini- > ITX > to Supermicro PDSBM-LN2 and OS from 8.4/i386 to 10.3-RELEASE- > p12/amd64 it > starts to work unstable. Most of the time it keeps time pretty well > with > offset less than 1-2 us, but once a few hours pll frequency jumps, > then > clock drifts. After passing calibration interval time (256s) > frequency > returns back to normal, then, after appropriate time, clock > stabilizes > again. Excerpt from loopstats: > 57734 37624.525 -0.000000955 0.120 0.000000588 0.000211 4 > 57734 37640.526 0.000000319 0.120 0.000000506 0.000198 4 > 57734 37656.526 -0.000000789 0.120 0.001081214 0.000185 4 > 57734 37672.526 -0.000398921 100.120 0.000154630 35.355339 4 > 57734 37688.526 -0.001941140 100.120 0.000188374 33.071891 4 > 57734 37704.525 -0.003389196 100.120 0.000177488 30.935922 4 > 57734 37720.525 -0.004745689 100.120 0.000166147 28.937905 4 > 57734 37736.525 -0.006022007 100.120 0.000156269 27.068931 4 > 57734 37752.526 -0.007220430 100.120 0.000146663 25.320667 4 > 57734 37768.526 -0.008343331 100.120 0.000137805 23.685315 4 > 57734 37784.525 -0.009399651 100.120 0.000129406 22.155583 4 > 57734 37800.525 -0.010391390 100.120 0.000121937 20.724651 4 > 57734 37816.526 -0.011320293 100.120 0.000114053 19.386136 4 > 57734 37832.526 -0.012194902 100.120 0.000107191 18.134069 4 > 57734 37848.526 -0.013013037 100.120 0.000100035 16.962869 4 > 57734 37864.526 -0.013783932 100.120 0.000094497 15.867311 4 > 57734 37880.526 -0.014507271 100.120 0.000088691 14.842510 4 > 57734 37896.525 -0.015184384 100.120 0.000083266 13.883897 4 > 57734 37912.526 -0.015822296 100.120 0.000078249 12.987196 4 > 57734 37928.525 -0.016119704 0.122 0.000103405 37.383615 4 > 57734 37944.526 -0.015132723 0.122 0.000120509 34.969170 4 > 57734 37960.526 -0.014207941 0.122 0.000113355 32.710663 4 > 57734 37976.525 -0.013339661 0.122 0.000107051 30.598023 4 >  [snip] > 57734 40296.525 -0.000000337 0.122 0.000001621 0.002136 4 > 57734 40312.526 -0.000000980 0.122 0.000001635 0.001998 4 > > The change in pll frequency is usually 100ppm, but not always. For > today, > for example, it's 29ppm once, 69.3ppm once and 100ppm three times. > > Had tried three available timecounters: TSC-low, ACPI-fast, HPET. Had > changed eventtimer from HPET to LAPIC, kern.eventtimer.periodic from > 0 to 1. > All the changes are followed by service ntpd restart. > Also tried to change kern.hz from 1000 to 100. > Had even tried 11.0 on other, but the exactly same board. The > original > board has OCXO instead of quartz, but reconnecting the original > quartz > doesn't help. > > Didn't try another hardware and/or OS yet, the server isn't easy > reachable, > but, in lack of better ideas, will definitely try. > > > Kernel has stripped all unused drivers and options plus PPS_SYNC, > then > FFCLOCK added. All the additions: > options         IPSEC > options         GEOM_ELI > options         PPS_SYNC > options         FFCLOCK > > device          crypto > device          enc > device          pf > device          pflog > device          smbus > device          ichsmb > device          smb > device          coretemp > device          cpuctl > device          nvram > device          smbios > device          ipmi > device          aesni > > The relevant part of ntp.conf: > fudge  127.127.20.0 time2 0.6 flag1 1 flag2 0 flag3 1 > server 127.127.20.0 mode 2 minpoll 4 prefer > server    minpoll 8 iburst > restrict default limited kod nomodify notrap nopeer noquery > > rc.conf: > ntpd_program="/usr/local/sbin/ntpd" > ntpd_config="/etc/ntpd.conf" > ntpd_flags="-N -p /var/run/ntpd.pid -f /var/db/ntpd.drift" > ntpd_sync_on_start="YES" > > sysctl.conf (this change is also seems irrelevant, rebooting without > this > frequency correction changes nothing in the behaviour): > machdep.tsc_freq=2194498500     # pll freq offset change from 21.678 > to 0.120ppm > > $ sysctl kern.hz kern.timecounter kern.eventtimer > kern.hz: 1000 > kern.timecounter.tsc_shift: 1 > kern.timecounter.smp_tsc_adjust: 0 > kern.timecounter.smp_tsc: 1 > kern.timecounter.invariant_tsc: 1 > kern.timecounter.fast_gettime: 1 > kern.timecounter.tick: 1 > kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) i8254(0) > HPET(950) dummy(-1000000) > kern.timecounter.hardware: TSC-low > kern.timecounter.alloweddeviation: 5 > kern.timecounter.stepwarnings: 0 > kern.timecounter.tc.TSC-low.quality: 1000 > kern.timecounter.tc.TSC-low.frequency: 1097249250 > kern.timecounter.tc.TSC-low.counter: 2359573202 > kern.timecounter.tc.TSC-low.mask: 4294967295 > kern.timecounter.tc.ACPI-fast.quality: 900 > kern.timecounter.tc.ACPI-fast.frequency: 3579545 > kern.timecounter.tc.ACPI-fast.counter: 9238615 > kern.timecounter.tc.ACPI-fast.mask: 16777215 > kern.timecounter.tc.i8254.quality: 0 > kern.timecounter.tc.i8254.frequency: 1193182 > kern.timecounter.tc.i8254.counter: 9906 > kern.timecounter.tc.i8254.mask: 65535 > kern.timecounter.tc.HPET.quality: 950 > kern.timecounter.tc.HPET.frequency: 14318180 > kern.timecounter.tc.HPET.counter: 2305610093 > kern.timecounter.tc.HPET.mask: 4294967295 > kern.eventtimer.periodic: 0 > kern.eventtimer.timer: HPET > kern.eventtimer.idletick: 0 > kern.eventtimer.singlemul: 2 > kern.eventtimer.choice: HPET(450) HPET1(440) HPET2(440) LAPIC(400) > i8254(100) RTC(0) > kern.eventtimer.et.i8254.quality: 100 > kern.eventtimer.et.i8254.frequency: 1193182 > kern.eventtimer.et.i8254.flags: 1 > kern.eventtimer.et.RTC.quality: 0 > kern.eventtimer.et.RTC.frequency: 32768 > kern.eventtimer.et.RTC.flags: 17 > kern.eventtimer.et.HPET2.quality: 440 > kern.eventtimer.et.HPET2.frequency: 14318180 > kern.eventtimer.et.HPET2.flags: 3 > kern.eventtimer.et.HPET1.quality: 440 > kern.eventtimer.et.HPET1.frequency: 14318180 > kern.eventtimer.et.HPET1.flags: 3 > kern.eventtimer.et.HPET.quality: 450 > kern.eventtimer.et.HPET.frequency: 14318180 > kern.eventtimer.et.HPET.flags: 3 > kern.eventtimer.et.LAPIC.quality: 400 > kern.eventtimer.et.LAPIC.frequency: 99749970 > kern.eventtimer.et.LAPIC.flags: 15 > $  > > Any hints ? > > Thank you, > Hrant > Very strange, I've never seen behavior like that.  You're using ntpd from ports, is it the latest version? How are you feeding the PPS signal to the system?  Do you know how wide the PPS pulse is?  I'm wondering if the driver is occasionally missing an edge of a narrow pulse, although an occasional bad measurement should get weeded out by ntpd's refclock median filter.  If the pulse is wider than a few microseconds the whole theory falls apart anyway. Anyway, I'm a bit focused on the PPS because there were changes to the serial (uart) PPS capture between 8.4 and 10.x, and I'm responsible for some of them. :) -- Ian From owner-freebsd-hackers@freebsd.org Mon Dec 12 19:04:10 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8FD7FC72E73 for ; Mon, 12 Dec 2016 19:04:10 +0000 (UTC) (envelope-from hrant@dadivanyan.net) Received: from pandora.amnic.net (pandora.amnic.net [IPv6:2001:67c:21c:a610::11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FE0615EF; Mon, 12 Dec 2016 19:04:10 +0000 (UTC) (envelope-from hrant@dadivanyan.net) Received: from ran by pandora.amnic.net with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1cGVtQ-000Acm-7c; Mon, 12 Dec 2016 23:04:08 +0400 Subject: Re: system time instability In-Reply-To: <1481556581.1889.322.camel@freebsd.org> To: Ian Lepore Date: Mon, 12 Dec 2016 23:04:08 +0400 (AMT) From: Hrant Dadivanyan CC: freebsd-hackers@freebsd.org X-PGP: https://amnic.net/pgpkeys/hrant.asc X-NCC-RegID: am.isoc X-Mailer: ELM [version 2.4ME+ PL127 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="UTF-8" Message-Id: X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 19:04:10 -0000 [ Charset ISO-8859-1 converted... ] > On Mon, 2016-12-12 at 17:23 +0400, Hrant Dadivanyan wrote: > > Hello, > > > > After upgrade of stratum 1 ntp server hardware from a Via EPIA Mini- > > ITX > > to Supermicro PDSBM-LN2 and OS from 8.4/i386 to 10.3-RELEASE- > > p12/amd64 it > > starts to work unstable. Most of the time it keeps time pretty well > > with > > offset less than 1-2 us, but once a few hours pll frequency jumps, > > then > > clock drifts. After passing calibration interval time (256s) > > frequency > > returns back to normal, then, after appropriate time, clock > > stabilizes > > again. Excerpt from loopstats: > > 57734 37624.525 -0.000000955 0.120 0.000000588 0.000211 4 > > 57734 37640.526 0.000000319 0.120 0.000000506 0.000198 4 > > 57734 37656.526 -0.000000789 0.120 0.001081214 0.000185 4 > > 57734 37672.526 -0.000398921 100.120 0.000154630 35.355339 4 > > 57734 37688.526 -0.001941140 100.120 0.000188374 33.071891 4 > > 57734 37704.525 -0.003389196 100.120 0.000177488 30.935922 4 > > 57734 37720.525 -0.004745689 100.120 0.000166147 28.937905 4 > > 57734 37736.525 -0.006022007 100.120 0.000156269 27.068931 4 > > 57734 37752.526 -0.007220430 100.120 0.000146663 25.320667 4 > > 57734 37768.526 -0.008343331 100.120 0.000137805 23.685315 4 > > 57734 37784.525 -0.009399651 100.120 0.000129406 22.155583 4 > > 57734 37800.525 -0.010391390 100.120 0.000121937 20.724651 4 > > 57734 37816.526 -0.011320293 100.120 0.000114053 19.386136 4 > > 57734 37832.526 -0.012194902 100.120 0.000107191 18.134069 4 > > 57734 37848.526 -0.013013037 100.120 0.000100035 16.962869 4 > > 57734 37864.526 -0.013783932 100.120 0.000094497 15.867311 4 > > 57734 37880.526 -0.014507271 100.120 0.000088691 14.842510 4 > > 57734 37896.525 -0.015184384 100.120 0.000083266 13.883897 4 > > 57734 37912.526 -0.015822296 100.120 0.000078249 12.987196 4 > > 57734 37928.525 -0.016119704 0.122 0.000103405 37.383615 4 > > 57734 37944.526 -0.015132723 0.122 0.000120509 34.969170 4 > > 57734 37960.526 -0.014207941 0.122 0.000113355 32.710663 4 > > 57734 37976.525 -0.013339661 0.122 0.000107051 30.598023 4 > >  [snip] > > 57734 40296.525 -0.000000337 0.122 0.000001621 0.002136 4 > > 57734 40312.526 -0.000000980 0.122 0.000001635 0.001998 4 > > > > The change in pll frequency is usually 100ppm, but not always. For > > today, > > for example, it's 29ppm once, 69.3ppm once and 100ppm three times. > > > > Had tried three available timecounters: TSC-low, ACPI-fast, HPET. Had > > changed eventtimer from HPET to LAPIC, kern.eventtimer.periodic from > > 0 to 1. > > All the changes are followed by service ntpd restart. > > Also tried to change kern.hz from 1000 to 100. > > Had even tried 11.0 on other, but the exactly same board. The > > original > > board has OCXO instead of quartz, but reconnecting the original > > quartz > > doesn't help. > > > > Didn't try another hardware and/or OS yet, the server isn't easy > > reachable, > > but, in lack of better ideas, will definitely try. > > > > > > Kernel has stripped all unused drivers and options plus PPS_SYNC, > > then > > FFCLOCK added. All the additions: > > options         IPSEC > > options         GEOM_ELI > > options         PPS_SYNC > > options         FFCLOCK > > > > device          crypto > > device          enc > > device          pf > > device          pflog > > device          smbus > > device          ichsmb > > device          smb > > device          coretemp > > device          cpuctl > > device          nvram > > device          smbios > > device          ipmi > > device          aesni > > > > The relevant part of ntp.conf: > > fudge  127.127.20.0 time2 0.6 flag1 1 flag2 0 flag3 1 > > server 127.127.20.0 mode 2 minpoll 4 prefer > > server    minpoll 8 iburst > > restrict default limited kod nomodify notrap nopeer noquery > > > > rc.conf: > > ntpd_program="/usr/local/sbin/ntpd" > > ntpd_config="/etc/ntpd.conf" > > ntpd_flags="-N -p /var/run/ntpd.pid -f /var/db/ntpd.drift" > > ntpd_sync_on_start="YES" > > > > sysctl.conf (this change is also seems irrelevant, rebooting without > > this > > frequency correction changes nothing in the behaviour): > > machdep.tsc_freq=2194498500     # pll freq offset change from 21.678 > > to 0.120ppm > > > > $ sysctl kern.hz kern.timecounter kern.eventtimer > > kern.hz: 1000 > > kern.timecounter.tsc_shift: 1 > > kern.timecounter.smp_tsc_adjust: 0 > > kern.timecounter.smp_tsc: 1 > > kern.timecounter.invariant_tsc: 1 > > kern.timecounter.fast_gettime: 1 > > kern.timecounter.tick: 1 > > kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) i8254(0) > > HPET(950) dummy(-1000000) > > kern.timecounter.hardware: TSC-low > > kern.timecounter.alloweddeviation: 5 > > kern.timecounter.stepwarnings: 0 > > kern.timecounter.tc.TSC-low.quality: 1000 > > kern.timecounter.tc.TSC-low.frequency: 1097249250 > > kern.timecounter.tc.TSC-low.counter: 2359573202 > > kern.timecounter.tc.TSC-low.mask: 4294967295 > > kern.timecounter.tc.ACPI-fast.quality: 900 > > kern.timecounter.tc.ACPI-fast.frequency: 3579545 > > kern.timecounter.tc.ACPI-fast.counter: 9238615 > > kern.timecounter.tc.ACPI-fast.mask: 16777215 > > kern.timecounter.tc.i8254.quality: 0 > > kern.timecounter.tc.i8254.frequency: 1193182 > > kern.timecounter.tc.i8254.counter: 9906 > > kern.timecounter.tc.i8254.mask: 65535 > > kern.timecounter.tc.HPET.quality: 950 > > kern.timecounter.tc.HPET.frequency: 14318180 > > kern.timecounter.tc.HPET.counter: 2305610093 > > kern.timecounter.tc.HPET.mask: 4294967295 > > kern.eventtimer.periodic: 0 > > kern.eventtimer.timer: HPET > > kern.eventtimer.idletick: 0 > > kern.eventtimer.singlemul: 2 > > kern.eventtimer.choice: HPET(450) HPET1(440) HPET2(440) LAPIC(400) > > i8254(100) RTC(0) > > kern.eventtimer.et.i8254.quality: 100 > > kern.eventtimer.et.i8254.frequency: 1193182 > > kern.eventtimer.et.i8254.flags: 1 > > kern.eventtimer.et.RTC.quality: 0 > > kern.eventtimer.et.RTC.frequency: 32768 > > kern.eventtimer.et.RTC.flags: 17 > > kern.eventtimer.et.HPET2.quality: 440 > > kern.eventtimer.et.HPET2.frequency: 14318180 > > kern.eventtimer.et.HPET2.flags: 3 > > kern.eventtimer.et.HPET1.quality: 440 > > kern.eventtimer.et.HPET1.frequency: 14318180 > > kern.eventtimer.et.HPET1.flags: 3 > > kern.eventtimer.et.HPET.quality: 450 > > kern.eventtimer.et.HPET.frequency: 14318180 > > kern.eventtimer.et.HPET.flags: 3 > > kern.eventtimer.et.LAPIC.quality: 400 > > kern.eventtimer.et.LAPIC.frequency: 99749970 > > kern.eventtimer.et.LAPIC.flags: 15 > > $  > > > > Any hints ? > > > > Thank you, > > Hrant > > > > Very strange, I've never seen behavior like that.  You're using ntpd > from ports, is it the latest version? > Yes, it's 4.2.8p9_1 from ports. > How are you feeding the PPS signal to the system?  Do you know how wide > the PPS pulse is?  I'm wondering if the driver is occasionally missing > an edge of a narrow pulse, although an occasional bad measurement > should get weeded out by ntpd's refclock median filter.  If the pulse > is wider than a few microseconds the whole theory falls apart anyway. > Pulse width is 100 ms, receiver is Garmin GPS 18x LVC. Actually I've replaced reciver as well. The cable is too long 12-13 meters and there was badformat (I guess CRC) errors, when setup 4-5 years ago. I've used CAT5 cable with PPS and Rx wires twised to ground and 74LS245 bus driver close to GPS receiver to amplify signal. It's not a real amplifier, but works fine there for years and these errors gone. There are also a few per a hour: kernel reports TIME_ERROR: 0x2307: PPS Time Sync wanted but PPS Jitter exceeded errors in the logs, so it looks like the signal is not okay anyway. > Anyway, I'm a bit focused on the PPS because there were changes to the > serial (uart) PPS capture between 8.4 and 10.x, and I'm responsible for > some of them. :) > Now, when you ask, I start to suspect PPS delivery to uart again - cable and amplifier, but can't understand how the 100ppm error fits into that. Thank you, Hrant > -- Ian > -- Hrant Dadivanyan (aka Ran d'Adi) hrant(at)dadivanyan.net /* "Feci quod potui, faciant meliora potentes." */ ran(at)psg.com From owner-freebsd-hackers@freebsd.org Mon Dec 12 19:18:53 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EAEF3C735D1 for ; Mon, 12 Dec 2016 19:18:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 802AACDB; Mon, 12 Dec 2016 19:18:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBCJIWmZ015083 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 12 Dec 2016 21:18:32 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBCJIWmZ015083 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBCJIWYt015082; Mon, 12 Dec 2016 21:18:32 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 12 Dec 2016 21:18:32 +0200 From: Konstantin Belousov To: Hrant Dadivanyan Cc: Ian Lepore , freebsd-hackers@freebsd.org Subject: Re: system time instability Message-ID: <20161212191832.GN54029@kib.kiev.ua> References: <1481556581.1889.322.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 19:18:54 -0000 On Mon, Dec 12, 2016 at 11:04:08PM +0400, Hrant Dadivanyan wrote: > Now, when you ask, I start to suspect PPS delivery to uart again - cable > and amplifier, but can't understand how the 100ppm error fits into that. If you disable PPS sync in ntp config, does the machine keep time adequately ? There might be relatively long pauses when system management mode handlers do something in response to hw events. E.g. if you have USB emulation of AT keyboard enabled in BIOS, try to disable that. And update the BIOS. From owner-freebsd-hackers@freebsd.org Mon Dec 12 19:34:02 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 96D4EC741CD for ; Mon, 12 Dec 2016 19:34:02 +0000 (UTC) (envelope-from hrant@dadivanyan.net) Received: from pandora.amnic.net (pandora.amnic.net [IPv6:2001:67c:21c:a610::11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4EAF5DB; Mon, 12 Dec 2016 19:34:02 +0000 (UTC) (envelope-from hrant@dadivanyan.net) Received: from ran by pandora.amnic.net with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1cGWMJ-000B5E-Mm; Mon, 12 Dec 2016 23:33:59 +0400 Subject: Re: system time instability In-Reply-To: <20161212191832.GN54029@kib.kiev.ua> To: Konstantin Belousov Date: Mon, 12 Dec 2016 23:33:59 +0400 (AMT) From: Hrant Dadivanyan CC: Ian Lepore , freebsd-hackers@freebsd.org X-PGP: https://amnic.net/pgpkeys/hrant.asc X-NCC-RegID: am.isoc X-Mailer: ELM [version 2.4ME+ PL127 (25)] MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="ELM1481571239-41986-0_" Content-Transfer-Encoding: 7bit Message-Id: X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 19:34:02 -0000 --ELM1481571239-41986-0_ Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="UTF-8" > On Mon, Dec 12, 2016 at 11:04:08PM +0400, Hrant Dadivanyan wrote: > > Now, when you ask, I start to suspect PPS delivery to uart again - cable > > and amplifier, but can't understand how the 100ppm error fits into that. > > If you disable PPS sync in ntp config, does the machine keep time adequately ? > Thanks for reminding - yes, I've tried this as well, the issue persists. So uart shouldn't be in charge. > There might be relatively long pauses when system management mode handlers > do something in response to hw events. E.g. if you have USB emulation of > AT keyboard enabled in BIOS, try to disable that. And update the BIOS. The USB is switched off in the BIOS. I've removed all changes in sysctl.conf and nice flag from ntpd, recompiled kernel as following: include GENERIC options PPS_SYNC device pf device pflog and started over. Dmesg is attached. Thank you, Hrant -- Hrant Dadivanyan (aka Ran d'Adi) hrant(at)dadivanyan.net /* "Feci quod potui, faciant meliora potentes." */ ran(at)psg.com --ELM1481571239-41986-0_ Content-Transfer-Encoding: 7bit Content-Type: text/plain Content-Disposition: attachment; filename=dmesg Content-Description: Copyright (c) 1992-2016 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.3-RELEASE-p14 #0 r309851: Mon Dec 12 19:38:52 AMT 2016 ran@ntp.amnic.net:/usr/obj/usr/src/sys/NTP amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 CPU: Intel(R) Core(TM)2 Duo CPU E4500 @ 2.20GHz (2194.55-MHz K8-class CPU) Origin="GenuineIntel" Id=0x6fd Family=0x6 Model=0xf Stepping=13 Features=0xbfebfbff Features2=0xe39d AMD Features=0x20100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 2147483648 (2048 MB) avail memory = 2040201216 (1945 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 random: initialized ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) cpu0: on acpi0 cpu1: on acpi0 hpet0: iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 450 Event timer "HPET1" frequency 14318180 Hz quality 440 Event timer "HPET2" frequency 14318180 Hz quality 440 atrtc0: port 0x70-0x71 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 vgapci0: port 0x3030-0x3037 mem 0xd0000000-0xd00fffff,0xc0000000-0xcfffffff irq 16 at device 2.0 on pci0 agp0: on vgapci0 agp0: aperture size is 256M, detected 7676k stolen memory vgapci0: Boot video device pcib1: irq 16 at device 28.0 on pci0 pcib1: failed to allocate initial I/O port window: 0-0xfff pcib1: failed to allocate initial memory window: 0-0xfffff pcib1: failed to allocate initial prefetch window: 0-0xfffff pci2: on pcib1 pcib2: irq 16 at device 28.4 on pci0 pci6: on pcib2 em0: port 0x4000-0x401f mem 0xd0100000-0xd011ffff irq 16 at device 0.0 on pci6 em0: Using an MSI interrupt em0: Ethernet address: 00:30:48:93:70:34 pcib3: irq 17 at device 28.5 on pci0 pci7: on pcib3 em1: port 0x5000-0x501f mem 0xd0200000-0xd021ffff irq 17 at device 0.0 on pci7 em1: Using an MSI interrupt em1: Ethernet address: 00:30:48:93:70:35 pcib4: at device 30.0 on pci0 pci8: on pcib4 isab0: at device 31.0 on pci0 isa0: on isab0 ahci0: port 0x3048-0x304f,0x303c-0x303f,0x3040-0x3047,0x3038-0x303b,0x3020-0x302f mem 0xd0500000-0xd05003ff irq 17 at device 31.2 on pci0 ahci0: AHCI v1.10 with 4 3Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 acpi_button0: on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 orm0: at iomem 0xc0000-0xcafff,0xcb000-0xcf7ff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] ppc0: cannot reserve I/O port range est0: on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr b270b2706000b27 device_attach: est0 attach returned 6 est1: on cpu1 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr b270b2706000b27 device_attach: est1 attach returned 6 random: unblocking device. Timecounters tick every 1.000 msec ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: ATA-7 SATA 2.x device ada0: Serial Number WD-WCAP94091954 ada0: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada0: Command Queueing enabled ada0: 152627MB (312581808 512 byte sectors) ada0: Previously was known as ad4 ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 ada1: ATA-7 SATA 2.x device ada1: Serial Number WD-WCAP94085021 ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada1: Command Queueing enabled ada1: 152627MB (312581808 512 byte sectors) ada1: Previously was known as ad6 GEOM_RAID: Intel-bbb292a3: Array Intel-bbb292a3 created. GEOM_RAID: Intel-bbb292a3: Disk ada0 state changed from NONE to ACTIVE. GEOM_RAID: Intel-bbb292a3: Subdisk Volume0:0-ada0 state changed from NONE to ACTIVE. GEOM_RAID: Intel-bbb292a3: Disk ada1 state changed from NONE to ACTIVE. GEOM_RAID: Intel-bbb292a3: Subdisk Volume0:1-ada1 state changed from NONE to ACTIVE. GEOM_RAID: Intel-bbb292a3: Array started. GEOM_RAID: Intel-bbb292a3: Volume Volume0 state changed from STARTING to OPTIMAL. GEOM_RAID: Intel-bbb292a3: Provider raid/r0 for volume Volume0 created. SMP: AP CPU #1 Launched! Timecounter "TSC-low" frequency 1097274299 Hz quality 1000 Trying to mount root from ufs:/dev/raid/r0p2 [rw]... em0: link state changed to UP pflog0: promiscuous mode enabled --ELM1481571239-41986-0_-- From owner-freebsd-hackers@freebsd.org Mon Dec 12 19:34:18 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2AD8EC74214 for ; Mon, 12 Dec 2016 19:34:18 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0FA5A1EA for ; Mon, 12 Dec 2016 19:34:17 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: ff445634-c0a1-11e6-9f67-d3961ed5a660 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id ff445634-c0a1-11e6-9f67-d3961ed5a660; Mon, 12 Dec 2016 19:34:29 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id uBCJYFTu008995; Mon, 12 Dec 2016 12:34:15 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1481571255.1889.329.camel@freebsd.org> Subject: Re: system time instability From: Ian Lepore To: Hrant Dadivanyan Cc: freebsd-hackers@freebsd.org Date: Mon, 12 Dec 2016 12:34:15 -0700 In-Reply-To: References: Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 19:34:18 -0000 On Mon, 2016-12-12 at 23:04 +0400, Hrant Dadivanyan wrote: > [ Charset ISO-8859-1 converted... ] > > > > On Mon, 2016-12-12 at 17:23 +0400, Hrant Dadivanyan wrote: > > > > > > Hello, > > > > > > After upgrade of stratum 1 ntp server hardware from a Via EPIA > > > Mini- > > > ITX > > > to Supermicro PDSBM-LN2 and OS from 8.4/i386 to 10.3-RELEASE- > > > p12/amd64 it > > > starts to work unstable. Most of the time it keeps time pretty > > > well > > > with > > > offset less than 1-2 us, but once a few hours pll frequency > > > jumps, > > > then > > > clock drifts. After passing calibration interval time (256s) > > > frequency > > > returns back to normal, then, after appropriate time, clock > > > stabilizes > > > again. Excerpt from loopstats: > > > 57734 37624.525 -0.000000955 0.120 0.000000588 0.000211 4 > > > 57734 37640.526 0.000000319 0.120 0.000000506 0.000198 4 > > > 57734 37656.526 -0.000000789 0.120 0.001081214 0.000185 4 > > > 57734 37672.526 -0.000398921 100.120 0.000154630 35.355339 4 > > > 57734 37688.526 -0.001941140 100.120 0.000188374 33.071891 4 > > > 57734 37704.525 -0.003389196 100.120 0.000177488 30.935922 4 > > > 57734 37720.525 -0.004745689 100.120 0.000166147 28.937905 4 > > > 57734 37736.525 -0.006022007 100.120 0.000156269 27.068931 4 > > > 57734 37752.526 -0.007220430 100.120 0.000146663 25.320667 4 > > > 57734 37768.526 -0.008343331 100.120 0.000137805 23.685315 4 > > > 57734 37784.525 -0.009399651 100.120 0.000129406 22.155583 4 > > > 57734 37800.525 -0.010391390 100.120 0.000121937 20.724651 4 > > > 57734 37816.526 -0.011320293 100.120 0.000114053 19.386136 4 > > > 57734 37832.526 -0.012194902 100.120 0.000107191 18.134069 4 > > > 57734 37848.526 -0.013013037 100.120 0.000100035 16.962869 4 > > > 57734 37864.526 -0.013783932 100.120 0.000094497 15.867311 4 > > > 57734 37880.526 -0.014507271 100.120 0.000088691 14.842510 4 > > > 57734 37896.525 -0.015184384 100.120 0.000083266 13.883897 4 > > > 57734 37912.526 -0.015822296 100.120 0.000078249 12.987196 4 > > > 57734 37928.525 -0.016119704 0.122 0.000103405 37.383615 4 > > > 57734 37944.526 -0.015132723 0.122 0.000120509 34.969170 4 > > > 57734 37960.526 -0.014207941 0.122 0.000113355 32.710663 4 > > > 57734 37976.525 -0.013339661 0.122 0.000107051 30.598023 4 > > >  [snip] > > > 57734 40296.525 -0.000000337 0.122 0.000001621 0.002136 4 > > > 57734 40312.526 -0.000000980 0.122 0.000001635 0.001998 4 > > > > > > The change in pll frequency is usually 100ppm, but not always. > > > For > > > today, > > > for example, it's 29ppm once, 69.3ppm once and 100ppm three > > > times. > > > > > > Had tried three available timecounters: TSC-low, ACPI-fast, HPET. > > > Had > > > changed eventtimer from HPET to LAPIC, kern.eventtimer.periodic > > > from > > > 0 to 1. > > > All the changes are followed by service ntpd restart. > > > Also tried to change kern.hz from 1000 to 100. > > > Had even tried 11.0 on other, but the exactly same board. The > > > original > > > board has OCXO instead of quartz, but reconnecting the original > > > quartz > > > doesn't help. > > > > > > Didn't try another hardware and/or OS yet, the server isn't easy > > > reachable, > > > but, in lack of better ideas, will definitely try. > > > > > > > > > Kernel has stripped all unused drivers and options plus PPS_SYNC, > > > then > > > FFCLOCK added. All the additions: > > > options         IPSEC > > > options         GEOM_ELI > > > options         PPS_SYNC > > > options         FFCLOCK > > > > > > device          crypto > > > device          enc > > > device          pf > > > device          pflog > > > device          smbus > > > device          ichsmb > > > device          smb > > > device          coretemp > > > device          cpuctl > > > device          nvram > > > device          smbios > > > device          ipmi > > > device          aesni > > > > > > The relevant part of ntp.conf: > > > fudge  127.127.20.0 time2 0.6 flag1 1 flag2 0 flag3 1 > > > server 127.127.20.0 mode 2 minpoll 4 prefer > > > server    minpoll 8 iburst > > > restrict default limited kod nomodify notrap nopeer noquery > > > > > > rc.conf: > > > ntpd_program="/usr/local/sbin/ntpd" > > > ntpd_config="/etc/ntpd.conf" > > > ntpd_flags="-N -p /var/run/ntpd.pid -f /var/db/ntpd.drift" > > > ntpd_sync_on_start="YES" > > > > > > sysctl.conf (this change is also seems irrelevant, rebooting > > > without > > > this > > > frequency correction changes nothing in the behaviour): > > > machdep.tsc_freq=2194498500     # pll freq offset change from > > > 21.678 > > > to 0.120ppm > > > > > > $ sysctl kern.hz kern.timecounter kern.eventtimer > > > kern.hz: 1000 > > > kern.timecounter.tsc_shift: 1 > > > kern.timecounter.smp_tsc_adjust: 0 > > > kern.timecounter.smp_tsc: 1 > > > kern.timecounter.invariant_tsc: 1 > > > kern.timecounter.fast_gettime: 1 > > > kern.timecounter.tick: 1 > > > kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) i8254(0) > > > HPET(950) dummy(-1000000) > > > kern.timecounter.hardware: TSC-low > > > kern.timecounter.alloweddeviation: 5 > > > kern.timecounter.stepwarnings: 0 > > > kern.timecounter.tc.TSC-low.quality: 1000 > > > kern.timecounter.tc.TSC-low.frequency: 1097249250 > > > kern.timecounter.tc.TSC-low.counter: 2359573202 > > > kern.timecounter.tc.TSC-low.mask: 4294967295 > > > kern.timecounter.tc.ACPI-fast.quality: 900 > > > kern.timecounter.tc.ACPI-fast.frequency: 3579545 > > > kern.timecounter.tc.ACPI-fast.counter: 9238615 > > > kern.timecounter.tc.ACPI-fast.mask: 16777215 > > > kern.timecounter.tc.i8254.quality: 0 > > > kern.timecounter.tc.i8254.frequency: 1193182 > > > kern.timecounter.tc.i8254.counter: 9906 > > > kern.timecounter.tc.i8254.mask: 65535 > > > kern.timecounter.tc.HPET.quality: 950 > > > kern.timecounter.tc.HPET.frequency: 14318180 > > > kern.timecounter.tc.HPET.counter: 2305610093 > > > kern.timecounter.tc.HPET.mask: 4294967295 > > > kern.eventtimer.periodic: 0 > > > kern.eventtimer.timer: HPET > > > kern.eventtimer.idletick: 0 > > > kern.eventtimer.singlemul: 2 > > > kern.eventtimer.choice: HPET(450) HPET1(440) HPET2(440) > > > LAPIC(400) > > > i8254(100) RTC(0) > > > kern.eventtimer.et.i8254.quality: 100 > > > kern.eventtimer.et.i8254.frequency: 1193182 > > > kern.eventtimer.et.i8254.flags: 1 > > > kern.eventtimer.et.RTC.quality: 0 > > > kern.eventtimer.et.RTC.frequency: 32768 > > > kern.eventtimer.et.RTC.flags: 17 > > > kern.eventtimer.et.HPET2.quality: 440 > > > kern.eventtimer.et.HPET2.frequency: 14318180 > > > kern.eventtimer.et.HPET2.flags: 3 > > > kern.eventtimer.et.HPET1.quality: 440 > > > kern.eventtimer.et.HPET1.frequency: 14318180 > > > kern.eventtimer.et.HPET1.flags: 3 > > > kern.eventtimer.et.HPET.quality: 450 > > > kern.eventtimer.et.HPET.frequency: 14318180 > > > kern.eventtimer.et.HPET.flags: 3 > > > kern.eventtimer.et.LAPIC.quality: 400 > > > kern.eventtimer.et.LAPIC.frequency: 99749970 > > > kern.eventtimer.et.LAPIC.flags: 15 > > > $  > > > > > > Any hints ? > > > > > > Thank you, > > > Hrant > > > > > Very strange, I've never seen behavior like that.  You're using > > ntpd > > from ports, is it the latest version? > > > Yes, it's 4.2.8p9_1 from ports. > > > > > How are you feeding the PPS signal to the system?  Do you know how > > wide > > the PPS pulse is?  I'm wondering if the driver is occasionally > > missing > > an edge of a narrow pulse, although an occasional bad measurement > > should get weeded out by ntpd's refclock median filter.  If the > > pulse > > is wider than a few microseconds the whole theory falls apart > > anyway. > > > Pulse width is 100 ms, receiver is Garmin GPS 18x LVC. Actually I've > replaced reciver as well. The cable is too long 12-13 meters and > there was > badformat (I guess CRC) errors, when setup 4-5 years ago. I've used > CAT5 > cable with PPS and Rx wires twised to ground and 74LS245 bus driver > close > to GPS receiver to amplify signal. It's not a real amplifier, but > works > fine there for years and these errors gone. > There are also a few per a hour: > kernel reports TIME_ERROR: 0x2307: PPS Time Sync wanted but PPS > Jitter exceeded > errors in the logs, so it looks like the signal is not okay anyway. > > > > > Anyway, I'm a bit focused on the PPS because there were changes to > > the > > serial (uart) PPS capture between 8.4 and 10.x, and I'm responsible > > for > > some of them. :) > > > Now, when you ask, I start to suspect PPS delivery to uart again - > cable > and amplifier, but can't understand how the 100ppm error fits into > that. > > Thank you, > Hrant Hmm.  But the one part of the system that didn't change (even if it was a bit bad all along) was delivery of the PPS signal.  Maybe the uart on the new hardware is more sensitive to the bad signaling. If you have a usb-serial adapter available, you could try using it for the PPS instead of the uart port on the new motherboard.  A USB serial adapter performs surprisingly well for PPS input, with a fixed latency that averages around 60 microseconds and fairly small jitter. The uart(4) manpage now has some information on configuring PPS inputs for traditional motherboard-style rs232, and ucom(4) has the info for usb-serial adapters. -- Ian From owner-freebsd-hackers@freebsd.org Mon Dec 12 19:44:09 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E76B9C7458C for ; Mon, 12 Dec 2016 19:44:09 +0000 (UTC) (envelope-from hrant@dadivanyan.net) Received: from pandora.amnic.net (pandora.amnic.net [IPv6:2001:67c:21c:a610::11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A783DA86; Mon, 12 Dec 2016 19:44:09 +0000 (UTC) (envelope-from hrant@dadivanyan.net) Received: from ran by pandora.amnic.net with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1cGWW7-000BGN-QA; Mon, 12 Dec 2016 23:44:07 +0400 Subject: Re: system time instability In-Reply-To: <1481571255.1889.329.camel@freebsd.org> To: Ian Lepore Date: Mon, 12 Dec 2016 23:44:07 +0400 (AMT) From: Hrant Dadivanyan CC: freebsd-hackers@freebsd.org X-PGP: https://amnic.net/pgpkeys/hrant.asc X-NCC-RegID: am.isoc X-Mailer: ELM [version 2.4ME+ PL127 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="UTF-8" Message-Id: X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 19:44:10 -0000 [ Charset ISO-8859-1 converted... ] > On Mon, 2016-12-12 at 23:04 +0400, Hrant Dadivanyan wrote: > > [ Charset ISO-8859-1 converted... ] > > > > > > On Mon, 2016-12-12 at 17:23 +0400, Hrant Dadivanyan wrote: > > > > > > > > Hello, > > > > > > > > After upgrade of stratum 1 ntp server hardware from a Via EPIA > > > > Mini- > > > > ITX > > > > to Supermicro PDSBM-LN2 and OS from 8.4/i386 to 10.3-RELEASE- > > > > p12/amd64 it > > > > starts to work unstable. Most of the time it keeps time pretty > > > > well > > > > with > > > > offset less than 1-2 us, but once a few hours pll frequency > > > > jumps, > > > > then > > > > clock drifts. After passing calibration interval time (256s) > > > > frequency > > > > returns back to normal, then, after appropriate time, clock > > > > stabilizes > > > > again. Excerpt from loopstats: > > > > 57734 37624.525 -0.000000955 0.120 0.000000588 0.000211 4 > > > > 57734 37640.526 0.000000319 0.120 0.000000506 0.000198 4 > > > > 57734 37656.526 -0.000000789 0.120 0.001081214 0.000185 4 > > > > 57734 37672.526 -0.000398921 100.120 0.000154630 35.355339 4 > > > > 57734 37688.526 -0.001941140 100.120 0.000188374 33.071891 4 > > > > 57734 37704.525 -0.003389196 100.120 0.000177488 30.935922 4 > > > > 57734 37720.525 -0.004745689 100.120 0.000166147 28.937905 4 > > > > 57734 37736.525 -0.006022007 100.120 0.000156269 27.068931 4 > > > > 57734 37752.526 -0.007220430 100.120 0.000146663 25.320667 4 > > > > 57734 37768.526 -0.008343331 100.120 0.000137805 23.685315 4 > > > > 57734 37784.525 -0.009399651 100.120 0.000129406 22.155583 4 > > > > 57734 37800.525 -0.010391390 100.120 0.000121937 20.724651 4 > > > > 57734 37816.526 -0.011320293 100.120 0.000114053 19.386136 4 > > > > 57734 37832.526 -0.012194902 100.120 0.000107191 18.134069 4 > > > > 57734 37848.526 -0.013013037 100.120 0.000100035 16.962869 4 > > > > 57734 37864.526 -0.013783932 100.120 0.000094497 15.867311 4 > > > > 57734 37880.526 -0.014507271 100.120 0.000088691 14.842510 4 > > > > 57734 37896.525 -0.015184384 100.120 0.000083266 13.883897 4 > > > > 57734 37912.526 -0.015822296 100.120 0.000078249 12.987196 4 > > > > 57734 37928.525 -0.016119704 0.122 0.000103405 37.383615 4 > > > > 57734 37944.526 -0.015132723 0.122 0.000120509 34.969170 4 > > > > 57734 37960.526 -0.014207941 0.122 0.000113355 32.710663 4 > > > > 57734 37976.525 -0.013339661 0.122 0.000107051 30.598023 4 > > > >  [snip] > > > > 57734 40296.525 -0.000000337 0.122 0.000001621 0.002136 4 > > > > 57734 40312.526 -0.000000980 0.122 0.000001635 0.001998 4 > > > > > > > > The change in pll frequency is usually 100ppm, but not always. > > > > For > > > > today, > > > > for example, it's 29ppm once, 69.3ppm once and 100ppm three > > > > times. > > > > > > > > Had tried three available timecounters: TSC-low, ACPI-fast, HPET. > > > > Had > > > > changed eventtimer from HPET to LAPIC, kern.eventtimer.periodic > > > > from > > > > 0 to 1. > > > > All the changes are followed by service ntpd restart. > > > > Also tried to change kern.hz from 1000 to 100. > > > > Had even tried 11.0 on other, but the exactly same board. The > > > > original > > > > board has OCXO instead of quartz, but reconnecting the original > > > > quartz > > > > doesn't help. > > > > > > > > Didn't try another hardware and/or OS yet, the server isn't easy > > > > reachable, > > > > but, in lack of better ideas, will definitely try. > > > > > > > > > > > > Kernel has stripped all unused drivers and options plus PPS_SYNC, > > > > then > > > > FFCLOCK added. All the additions: > > > > options         IPSEC > > > > options         GEOM_ELI > > > > options         PPS_SYNC > > > > options         FFCLOCK > > > > > > > > device          crypto > > > > device          enc > > > > device          pf > > > > device          pflog > > > > device          smbus > > > > device          ichsmb > > > > device          smb > > > > device          coretemp > > > > device          cpuctl > > > > device          nvram > > > > device          smbios > > > > device          ipmi > > > > device          aesni > > > > > > > > The relevant part of ntp.conf: > > > > fudge  127.127.20.0 time2 0.6 flag1 1 flag2 0 flag3 1 > > > > server 127.127.20.0 mode 2 minpoll 4 prefer > > > > server    minpoll 8 iburst > > > > restrict default limited kod nomodify notrap nopeer noquery > > > > > > > > rc.conf: > > > > ntpd_program="/usr/local/sbin/ntpd" > > > > ntpd_config="/etc/ntpd.conf" > > > > ntpd_flags="-N -p /var/run/ntpd.pid -f /var/db/ntpd.drift" > > > > ntpd_sync_on_start="YES" > > > > > > > > sysctl.conf (this change is also seems irrelevant, rebooting > > > > without > > > > this > > > > frequency correction changes nothing in the behaviour): > > > > machdep.tsc_freq=2194498500     # pll freq offset change from > > > > 21.678 > > > > to 0.120ppm > > > > > > > > $ sysctl kern.hz kern.timecounter kern.eventtimer > > > > kern.hz: 1000 > > > > kern.timecounter.tsc_shift: 1 > > > > kern.timecounter.smp_tsc_adjust: 0 > > > > kern.timecounter.smp_tsc: 1 > > > > kern.timecounter.invariant_tsc: 1 > > > > kern.timecounter.fast_gettime: 1 > > > > kern.timecounter.tick: 1 > > > > kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) i8254(0) > > > > HPET(950) dummy(-1000000) > > > > kern.timecounter.hardware: TSC-low > > > > kern.timecounter.alloweddeviation: 5 > > > > kern.timecounter.stepwarnings: 0 > > > > kern.timecounter.tc.TSC-low.quality: 1000 > > > > kern.timecounter.tc.TSC-low.frequency: 1097249250 > > > > kern.timecounter.tc.TSC-low.counter: 2359573202 > > > > kern.timecounter.tc.TSC-low.mask: 4294967295 > > > > kern.timecounter.tc.ACPI-fast.quality: 900 > > > > kern.timecounter.tc.ACPI-fast.frequency: 3579545 > > > > kern.timecounter.tc.ACPI-fast.counter: 9238615 > > > > kern.timecounter.tc.ACPI-fast.mask: 16777215 > > > > kern.timecounter.tc.i8254.quality: 0 > > > > kern.timecounter.tc.i8254.frequency: 1193182 > > > > kern.timecounter.tc.i8254.counter: 9906 > > > > kern.timecounter.tc.i8254.mask: 65535 > > > > kern.timecounter.tc.HPET.quality: 950 > > > > kern.timecounter.tc.HPET.frequency: 14318180 > > > > kern.timecounter.tc.HPET.counter: 2305610093 > > > > kern.timecounter.tc.HPET.mask: 4294967295 > > > > kern.eventtimer.periodic: 0 > > > > kern.eventtimer.timer: HPET > > > > kern.eventtimer.idletick: 0 > > > > kern.eventtimer.singlemul: 2 > > > > kern.eventtimer.choice: HPET(450) HPET1(440) HPET2(440) > > > > LAPIC(400) > > > > i8254(100) RTC(0) > > > > kern.eventtimer.et.i8254.quality: 100 > > > > kern.eventtimer.et.i8254.frequency: 1193182 > > > > kern.eventtimer.et.i8254.flags: 1 > > > > kern.eventtimer.et.RTC.quality: 0 > > > > kern.eventtimer.et.RTC.frequency: 32768 > > > > kern.eventtimer.et.RTC.flags: 17 > > > > kern.eventtimer.et.HPET2.quality: 440 > > > > kern.eventtimer.et.HPET2.frequency: 14318180 > > > > kern.eventtimer.et.HPET2.flags: 3 > > > > kern.eventtimer.et.HPET1.quality: 440 > > > > kern.eventtimer.et.HPET1.frequency: 14318180 > > > > kern.eventtimer.et.HPET1.flags: 3 > > > > kern.eventtimer.et.HPET.quality: 450 > > > > kern.eventtimer.et.HPET.frequency: 14318180 > > > > kern.eventtimer.et.HPET.flags: 3 > > > > kern.eventtimer.et.LAPIC.quality: 400 > > > > kern.eventtimer.et.LAPIC.frequency: 99749970 > > > > kern.eventtimer.et.LAPIC.flags: 15 > > > > $  > > > > > > > > Any hints ? > > > > > > > > Thank you, > > > > Hrant > > > > > > > Very strange, I've never seen behavior like that.  You're using > > > ntpd > > > from ports, is it the latest version? > > > > > Yes, it's 4.2.8p9_1 from ports. > > > > > > > > How are you feeding the PPS signal to the system?  Do you know how > > > wide > > > the PPS pulse is?  I'm wondering if the driver is occasionally > > > missing > > > an edge of a narrow pulse, although an occasional bad measurement > > > should get weeded out by ntpd's refclock median filter.  If the > > > pulse > > > is wider than a few microseconds the whole theory falls apart > > > anyway. > > > > > Pulse width is 100 ms, receiver is Garmin GPS 18x LVC. Actually I've > > replaced reciver as well. The cable is too long 12-13 meters and > > there was > > badformat (I guess CRC) errors, when setup 4-5 years ago. I've used > > CAT5 > > cable with PPS and Rx wires twised to ground and 74LS245 bus driver > > close > > to GPS receiver to amplify signal. It's not a real amplifier, but > > works > > fine there for years and these errors gone. > > There are also a few per a hour: > > kernel reports TIME_ERROR: 0x2307: PPS Time Sync wanted but PPS > > Jitter exceeded > > errors in the logs, so it looks like the signal is not okay anyway. > > > > > > > > Anyway, I'm a bit focused on the PPS because there were changes to > > > the > > > serial (uart) PPS capture between 8.4 and 10.x, and I'm responsible > > > for > > > some of them. :) > > > > > Now, when you ask, I start to suspect PPS delivery to uart again - > > cable > > and amplifier, but can't understand how the 100ppm error fits into > > that. > > > > Thank you, > > Hrant > > Hmm.  But the one part of the system that didn't change (even if it was > a bit bad all along) was delivery of the PPS signal.  Maybe the uart on > the new hardware is more sensitive to the bad signaling. > Yes, but Konstantin's question reminded me that I've tried it without PPS as well, with the same issue. > If you have a usb-serial adapter available, you could try using it for > the PPS instead of the uart port on the new motherboard.  A USB serial > adapter performs surprisingly well for PPS input, with a fixed latency > that averages around 60 microseconds and fairly small jitter. > > The uart(4) manpage now has some information on configuring PPS inputs > for traditional motherboard-style rs232, and ucom(4) has the info for > usb-serial adapters. > > -- Ian > -- Hrant Dadivanyan (aka Ran d'Adi) hrant(at)dadivanyan.net /* "Feci quod potui, faciant meliora potentes." */ ran(at)psg.com From owner-freebsd-hackers@freebsd.org Tue Dec 13 00:57:09 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 71C22C73FFE for ; Tue, 13 Dec 2016 00:57:09 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x242.google.com (mail-io0-x242.google.com [IPv6:2607:f8b0:4001:c06::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 37EF61108 for ; Tue, 13 Dec 2016 00:57:09 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x242.google.com with SMTP id p13so25342010ioi.0 for ; Mon, 12 Dec 2016 16:57:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=0/KZbHC0XIcX7G58UmHOKjTIMly7Ii8Vp2/ppD6A9aY=; b=LY+lKKo5/FcaVhzjXQN9Ku3wLWpj/0r61S114bEgG8Jb8hHE8++MyNGqiQYX1gCWdL 4mJirLj5bYy1aybYCHIl0Xnf1aYxKjft20wcXPx7qSFWxtLGbs7BrzaPtAKMH6hAipx2 6toBUmhrVWHgJ+sK7eVPnP8ludHTmmVhYErMsc+LQ0Myj+cd4XEiOmbGCucJEZZKzmwP +vc2SMYabRIufe9bkb8C22c/vHRTZ+qPAFNELedMEwpL1eoDLjXLGH+zmn048zuEXmDJ c/a+5xAApG8IXoRpNxyg2lSHabpOBo2ZeM9EfZuKl092dxlUONW4a9MG1nE2QjQcjHvQ /JTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=0/KZbHC0XIcX7G58UmHOKjTIMly7Ii8Vp2/ppD6A9aY=; b=GT8Dyx16b2ih2GRPWFNIj8OG/ab/LQDKQu2B2qAgTiNC4gxpsR7oeojf6rCfYgQwF1 0iH5lbrMXIfu2HtuI96j+uWeOFPYA3Xx+f4xmMdC7It41WH1vM7ACr+nvRy0e2fsAnsy 8JAEu5X3Y0GDZF0vQeS3rifjFnwPmCeCfuSfFy1sf9NMTlgMPawIWVv4nHOLOcj2oR7t kTZGtXM4DUlZ3AFcVkHQ1ytP41P02BR1lD1OS9Zg38n+8heVC3uL/9RO/TuH4TYK5CyH hqWuOGDxFO/B5MidyCyzizRLQ3qcjXabFl9MhOfQ1RtFfIqdhqjylFMNKdFu7R8VnPhm ADwQ== X-Gm-Message-State: AKaTC00iuhqQJ9zGICOKqWGAf1wMlJ8J2ZY2V5jjrKEdytgPwbNbmSCgkSx9+8kxNQPbxSxDDKGl5yXfgzuX4g== X-Received: by 10.107.132.74 with SMTP id g71mr84128075iod.19.1481590628435; Mon, 12 Dec 2016 16:57:08 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.31.199 with HTTP; Mon, 12 Dec 2016 16:57:07 -0800 (PST) X-Originating-IP: [69.53.245.200] In-Reply-To: References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> From: Warner Losh Date: Mon, 12 Dec 2016 17:57:07 -0700 X-Google-Sender-Auth: _w2hk19iUdtpBe7uHY3Rc99r0KA Message-ID: Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: Ed Schouten Cc: Jonathan Anderson , FreeBSD Hackers , Allan Jude Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 00:57:09 -0000 On Mon, Dec 12, 2016 at 6:34 AM, Ed Schouten wrote: > Hi Jonathan, > > 2016-12-11 21:12 GMT+01:00 Jonathan Anderson : >> Rather than doing lots of system calls to >> enumerate clearly-related sysctl values, it would be very cool to do one >> system call and then inspect the resulting data structure in memory. > > I'm not sure it's worth tackling this problem at this time. My current > version of the exporter is already pretty fast: > > $ time ./prometheus_sysctl_exporter | wc -l > 0.00 real 0.00 user 0.00 sys > 3162 > > If our sysctl tree would get substantially larger at some point, we > should of course revisit this. That sounds cool. I'd love to see what you came up with to get those results. While we're here... there's one thing I'd love to see from a hypothetical /dev/sysctl... When a sysctl value changes (or some subset), I'd like it to appear on a read-channel. That way, I could monitor temperature w/o needing to constantly poll for changes, for example. Warner From owner-freebsd-hackers@freebsd.org Tue Dec 13 05:28:23 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 665E0C7547B for ; Tue, 13 Dec 2016 05:28:23 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (vps.rulingia.com [103.243.244.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.rulingia.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F0003119C for ; Tue, 13 Dec 2016 05:28:21 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (ppp59-167-167-3.static.internode.on.net [59.167.167.3]) by vps.rulingia.com (8.15.2/8.15.2) with ESMTPS id uBD5S5PD080228 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Dec 2016 16:28:11 +1100 (AEDT) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.15.2/8.15.2) with ESMTPS id uBD5RxP9047311 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 13 Dec 2016 16:27:59 +1100 (AEDT) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.15.2/8.15.2/Submit) id uBD5RxAo047310; Tue, 13 Dec 2016 16:27:59 +1100 (AEDT) (envelope-from peter) Date: Tue, 13 Dec 2016 16:27:59 +1100 From: Peter Jeremy To: Warner Losh Cc: Ed Schouten , FreeBSD Hackers Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems Message-ID: <20161213052759.GP61036@server.rulingia.com> References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="Md/poaVZ8hnGTzuv" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.7.1 (2016-10-04) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 05:28:23 -0000 --Md/poaVZ8hnGTzuv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2016-Dec-12 17:57:07 -0700, Warner Losh wrote: >On Mon, Dec 12, 2016 at 6:34 AM, Ed Schouten wrote: >> $ time ./prometheus_sysctl_exporter | wc -l >> 0.00 real 0.00 user 0.00 sys >> 3162 >> >> If our sysctl tree would get substantially larger at some point, we >> should of course revisit this. > >That sounds cool. I'd love to see what you came up with to get those resul= ts. I think that's impressive as well. >While we're here... there's one thing I'd love to see from a >hypothetical /dev/sysctl... When a sysctl value changes (or some >subset), I'd like it to appear on a read-channel. That way, I could >monitor temperature w/o needing to constantly poll for changes, for >example. That sounds like a job for kqueue(2), though it would need to be plumbed through. The biggest problem I see is that nothing in the sysctl mechanism has any way of reporting that a value has changed and so consumers of that data should re-check. This means that you'd probably just wind up polling in the kernel. Note that Prometheus is designed around regularly polling the systems it's monitoring so this sort of functionality wouldn't benefit it. --=20 Peter Jeremy --Md/poaVZ8hnGTzuv Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJYT4beXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFRUIyOTg2QzMwNjcxRTc0RTY1QzIyN0Ux NkE1OTdBMEU0QTIwQjM0AAoJEBall6Dkogs0dh0QAIvLQ91q08Bq0xEMXJpcA0uD A4aXZSZXVfIZ2yIn3hLXwdp7WJHfzpgoeYMhAuv1dq2fmEu4Yt9JyY1mus9FCPPP Q8AauacE+vp9MIYspdwTIC/47xBC6D+VjIxUbjgMhKwdXRgqEOt6dWnIAXgzvyCy osP/lp6p0meIsItM1ymF5QHijXuEi343Iy+J0inMeofzdj4C/Hk66QAJ58xMl6aJ pVE3g8UdCEpDwMTsz3+eM7OzUCr+/Pse0OUAYJfwj6kJiTjSK9WfEXLuhiLFg9JY sM1VFlA+R7387+EWdfvooQLm7NcgsaRdATtj2Y1V0MINRONf1pCN7PWV2a4JPvbn dhtejm1c1GWpTgswmyy/GrHedS1wGValcCKAWfWIjGV9e1iMXpIAcuTEr3gxYaML YNIoOFU4+ZudyHkgg7my4JtuTRv+3kbpq2tPqGoWpZuxqQ+gaYN20EdPgm39sTPg XXHiDaJYvJSDHwaQyzKADpDaeuN9PXNoZHhP1drs6RtY6s9+deeWFiegwT9KaEI1 SOAe8g1hgZNvlYuwsBbLy+dgqigwCVARnUpXoDQKyuHnt1L2Tc+mAohvVK38T/91 6rwkETOzNs6miDURJLlbxUE5wVPWN0/0+S9o2vNh4dvHYyCIDMxoDASIigYmcnTV d8hIo2kkvKhEGE6ioa7v =9J41 -----END PGP SIGNATURE----- --Md/poaVZ8hnGTzuv-- From owner-freebsd-hackers@freebsd.org Tue Dec 13 05:33:23 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E0125C7586E for ; Tue, 13 Dec 2016 05:33:23 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22e.google.com (mail-io0-x22e.google.com [IPv6:2607:f8b0:4001:c06::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A6629184B for ; Tue, 13 Dec 2016 05:33:23 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22e.google.com with SMTP id d9so207167240ioe.0 for ; Mon, 12 Dec 2016 21:33:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=bXR2w1w3Ka3LiSXypnmcvKxRs7kgouIK05kTLi3pDZY=; b=h6vzeUZz9I0Hne8qnWnpT0JnExjEbqzTV6i8h1mFLi35N7MCiYz3uOOWQyziUfqaZ1 9gAnQzACuIN3WFP6dovhahZUN23gg+cthzBtSoCgdyJsJ+rDUNco/cn+JzFVxe7zSA8i /n7cqDmUCldNBcy1bbU0nB8/o5MG0oHxyYWeuHa4Q/uGi+yEa+EQUBpKO5OQX9DprUGX FqHIsahsEZeY93cwSStrAd3LxlTBwluZektYXzZUTBupuh2Zcj50LmoVLc0LLQSxqlEa jLcR+6N+PH6+lliw98bkJR6vFsXmacZ53Hd48gk92jZtMUV95aR+YVmK4EGOnPQuHzTI HO1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=bXR2w1w3Ka3LiSXypnmcvKxRs7kgouIK05kTLi3pDZY=; b=KK8Xs3Jt8oRsVqqn+EzMXvb04Eyq99RLAO1Ur0VFUr+lKUROu0gMiW9P2CKRBtkuhD RtHCOgdRvo6yFtsAUcJDP82IS3yj4grhdHJr3BWdL+AsdVtLzaDn7RlfTisDAfM5P2SU mgt3GnxS+IO/WSkRb2l5lyKKkdFu5ObEFRJDwqJZwc23zuwSt/7ejxS/c3t5UTMRbIeD XZ2oScp3agGvSjXJz6KvvrrjNDVtpKTaR3Ip7/xUykxgEHBGHoPS4h7k47Ub7AeHJAVh T5lmLICUiHXues5j3P7ciSWqPLVNSZn1fzPlQfD7h7U07uj7P8/9Iv3cUYCUO1hk9Vi6 o3uw== X-Gm-Message-State: AKaTC03x2Pko2ZSU0FUzLYp5YHy14HB/Y0qWn0iPly8zdjyIpcRWOf7T5Iyc4+x+qtof5fIVvjjP2aYiRHXThw== X-Received: by 10.36.84.138 with SMTP id t132mr971646ita.103.1481607203010; Mon, 12 Dec 2016 21:33:23 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.31.199 with HTTP; Mon, 12 Dec 2016 21:33:22 -0800 (PST) X-Originating-IP: [69.53.245.200] In-Reply-To: References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> <20161213052759.GP61036@server.rulingia.com> From: Warner Losh Date: Mon, 12 Dec 2016 22:33:22 -0700 X-Google-Sender-Auth: 3DBRxeCesdlcPptbC8F3BM8b-n8 Message-ID: Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: Peter Jeremy Cc: Warner Losh , Ed Schouten , FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 05:33:24 -0000 On Mon, Dec 12, 2016 at 10:29 PM, Warner Losh wrote: > >> On Dec 12, 2016, at 10:27 PM, Peter Jeremy wrote: >> >> On 2016-Dec-12 17:57:07 -0700, Warner Losh wrote: >>> On Mon, Dec 12, 2016 at 6:34 AM, Ed Schouten wrote: >>>> $ time ./prometheus_sysctl_exporter | wc -l >>>> 0.00 real 0.00 user 0.00 sys >>>> 3162 >>>> >>>> If our sysctl tree would get substantially larger at some point, we >>>> should of course revisit this. >>> >>> That sounds cool. I'd love to see what you came up with to get those results. >> >> I think that's impressive as well. >> >>> While we're here... there's one thing I'd love to see from a >>> hypothetical /dev/sysctl... When a sysctl value changes (or some >>> subset), I'd like it to appear on a read-channel. That way, I could >>> monitor temperature w/o needing to constantly poll for changes, for >>> example. >> >> That sounds like a job for kqueue(2), though it would need to be plumbed >> through. The biggest problem I see is that nothing in the sysctl mechanism >> has any way of reporting that a value has changed and so consumers of that >> data should re-check. This means that you'd probably just wind up polling >> in the kernel. > > It does if you had a read-channel :) Yea, forgot about the pointer case, you're right. This is a cool idea, but it can't work. Warner From owner-freebsd-hackers@freebsd.org Tue Dec 13 06:03:55 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4706EC7423B for ; Tue, 13 Dec 2016 06:03:55 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x236.google.com (mail-io0-x236.google.com [IPv6:2607:f8b0:4001:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0C053903 for ; Tue, 13 Dec 2016 06:03:55 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x236.google.com with SMTP id d9so208001374ioe.0 for ; Mon, 12 Dec 2016 22:03:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=4gOAHi2PjT8DOFltb/4jVbzWFPNgjGFlE3WJ7RLUDng=; b=rXqdtsGNDjSq9gDXXHXbZ07eroNX8InH92NZMbfdkJ2Qmy3yhAqYph8BWF8KjCgXMO TdWQwzMhp0P1Ag98qL4wVefJHNw2VbRI9dnNBu1K5IwJ5y+HX/lop/GkEAiDOZYUS0QQ g/NbLGtKQ8knLf7zG2unJ6QCBdDJX5MiiE/P602pvhIElau47ZjbKZJ50QpsmeDBiYbi CGZHP4eftIh1jzLnZPYSGVPlgVwR8P+U2pCot0sugjWliN7kBiOvGuAsi7EnjIXzr1S0 iJCfwiSykp1yN9NJ8PPPI0i3hTxoAJaucr7cIk5mtxW0aUA7f7l0NgPpnC7bYZMaqf1w lW4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=4gOAHi2PjT8DOFltb/4jVbzWFPNgjGFlE3WJ7RLUDng=; b=EIIhoOad0bf5B3a2Lid4E7KIfkKaLBjmnRvOeQToYVeewIygL4iUtWwYG1Cq3v+0r1 jF/EpcG70NeKmpEkeSxfY5hPmb8h35YcYrpHlZvtFVMrNBgydlip8KoHTdbhyYUODQIi z1Jh8Q73R5Tpq1T2nt5/yQ7JuyRcdd9qo3YrOMdKQPoYze/LUA0UaeIeKzjpWmiXxz4X I+5vZJDMBl40FKBacuhBXphCaW25m+PrL59Q1ojRRhuTOj0epqz5Kf+buCiqyU4gq4w6 YQBF6Uqd8RFV4y/bcSrlthTriTkq0VG1vYOQsgZm2abAlSoQAfu7/D4rwf3XjZfXJff9 k1zQ== X-Gm-Message-State: AKaTC03LL4EMrx6uGTJeS46+MovgvqzUcFtwwvmzJnvxfey3pmBqOFh4UTcnySXVBwKXbFuIr49ZmHT/z6GWlA== X-Received: by 10.107.154.14 with SMTP id c14mr68920576ioe.0.1481609034411; Mon, 12 Dec 2016 22:03:54 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.31.199 with HTTP; Mon, 12 Dec 2016 22:03:53 -0800 (PST) X-Originating-IP: [12.49.214.34] In-Reply-To: References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> <20161213052759.GP61036@server.rulingia.com> From: Warner Losh Date: Mon, 12 Dec 2016 23:03:53 -0700 X-Google-Sender-Auth: lBnUA6JEtzAA9FCOW31rz6yeFxI Message-ID: Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: Adrian Chadd Cc: Peter Jeremy , FreeBSD Hackers , Ed Schouten Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 06:03:55 -0000 On Mon, Dec 12, 2016 at 10:49 PM, Adrian Chadd wrote: > If only we had a mechanism for these kinds of sensors to register, so > the API they'd be using would be one that lead itself to edge/level > triggered comparisons, versus polling. > > Wait, didn't we /have/ one of those up for review a few years ago, and > it wasn't merged in? Didn't we have people willing to work with the folks that put this up when to address the defects in the review? Oh, wait, they never wanted to work to fix the problems, even with many offers of help to do so. My offer still stands, btw. Warner From owner-freebsd-hackers@freebsd.org Tue Dec 13 10:37:27 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E5227C72443 for ; Tue, 13 Dec 2016 10:37:27 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 63A6115F1; Tue, 13 Dec 2016 10:37:27 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBDAb3fu081767 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 13 Dec 2016 12:37:03 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBDAb3fu081767 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBDAb3RG081765; Tue, 13 Dec 2016 12:37:03 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 13 Dec 2016 12:37:03 +0200 From: Konstantin Belousov To: Hrant Dadivanyan Cc: Ian Lepore , freebsd-hackers@freebsd.org Subject: Re: system time instability Message-ID: <20161213103703.GR54029@kib.kiev.ua> References: <20161212191832.GN54029@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 10:37:28 -0000 On Mon, Dec 12, 2016 at 11:33:59PM +0400, Hrant Dadivanyan wrote: > > On Mon, Dec 12, 2016 at 11:04:08PM +0400, Hrant Dadivanyan wrote: > > > Now, when you ask, I start to suspect PPS delivery to uart again - cable > > > and amplifier, but can't understand how the 100ppm error fits into that. > > > > If you disable PPS sync in ntp config, does the machine keep time adequately ? > > > > Thanks for reminding - yes, I've tried this as well, the issue persists. > So uart shouldn't be in charge. > > > There might be relatively long pauses when system management mode handlers > > do something in response to hw events. E.g. if you have USB emulation of > > AT keyboard enabled in BIOS, try to disable that. And update the BIOS. > > The USB is switched off in the BIOS. I've removed all changes in sysctl.conf > and nice flag from ntpd, recompiled kernel as following: > include GENERIC > options PPS_SYNC > device pf > device pflog > and started over. Dmesg is attached. > Please show verbose dmesg. > CPU: Intel(R) Core(TM)2 Duo CPU E4500 @ 2.20GHz (2194.55-MHz K8-class CPU) This is relatively old CPU which is known to have some (minor) issues with interaction between power saving and cores. Try the following OS config: disable deep C states, allow only C1 (there might be some tweaks in BIOS, if possible, disable the Cn, n > 1, there too); use LAPIC for event timer (not HPET); re-check that you use RDTSC for the timecounter; do not enable powerd. You might also try the stable/11 kernel, which has more changes WRT C-states handling and PPS/ntp locking. From owner-freebsd-hackers@freebsd.org Tue Dec 13 05:29:48 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 21622C7564F for ; Tue, 13 Dec 2016 05:29:48 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pg0-x22a.google.com (mail-pg0-x22a.google.com [IPv6:2607:f8b0:400e:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E0B9A150F for ; Tue, 13 Dec 2016 05:29:47 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-pg0-x22a.google.com with SMTP id f188so43626105pgc.3 for ; Mon, 12 Dec 2016 21:29:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=subject:mime-version:from:in-reply-to:date:cc:message-id:references :to; bh=HcuZtrcP3wSorzbstyLJyVJUrJCSiGrEA3FFRl7Qv38=; b=Vqry6SopsE6LfdOi34ysEzQnrc/yy1vjEz4wHmMPqgPB26uuUCuV71FUMoxZnzzfkx luZd00mj/p6UKw1Cdbl7VjbWEvuxEEX995GCrH5AYtwQrtvSHhaGjMcBWtANV1zCaDLm 3znT2fGRDzzvaIhXeBEL9mAGswj4lnFBwn/0qYwgdfvBfrV2HfVfcnRThtXENzyF+g6I VcihgCZ6lS5ILPHiJD1+xLMk2tuymvErMipQUtq1fy5Hs7u4GSAKy7Y+dES5IdVN4mHB xxSvYhA4viRY1hBpPqeLAk8o3pLjlBEFlXkAzpobirV+zO8zH0t3EDuhlO0HnI0sT4z8 +sqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :message-id:references:to; bh=HcuZtrcP3wSorzbstyLJyVJUrJCSiGrEA3FFRl7Qv38=; b=R2M8/yl1n9GnBTnTOUmL4uKrQlb45l+Iyo6yViebfsAPK1vYMhgdw2XagdEExvJzBd YKrOmSjcEYq5VX8DQdoJroUKZgTME183I+kvMZIw7RKuL2aUxsqibRG/62dhEUfcHjeN 04jgg1MnpfK2WSBlnexcVtDOR42DC1nChGitSa48jU2TWyNCWxHf3jiccHkHjQ7ax5V5 vUFniWa5cZjAObopKmFoZWiBhwnzO2OWAenqEZDqWjHyiz+TDHN/7wJX8tYTYeDunm7+ yslN0N3D4yNAs8KiR47xCMHNN/5HHiMatHeOcQIbmA7fW2kxxb8kBjtIFqrgZhdL2Ucj UqYA== X-Gm-Message-State: AKaTC01ojljOMnnVd0d/cHLAuT+ORJvvSmOfLSXxpvL4bndiM3z5rbW8vz/HyafS2lZpuQ== X-Received: by 10.84.217.216 with SMTP id d24mr193732355plj.10.1481606987405; Mon, 12 Dec 2016 21:29:47 -0800 (PST) Received: from bhwl-mwheet2.corp.netflix.com ([69.53.245.200]) by smtp.gmail.com with ESMTPSA id s65sm79307385pgb.25.2016.12.12.21.29.46 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 12 Dec 2016 21:29:46 -0800 (PST) Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: multipart/signed; boundary="Apple-Mail=_15F5C561-94BD-43D4-A6BE-9BDA98F3F57C"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail From: Warner Losh In-Reply-To: <20161213052759.GP61036@server.rulingia.com> Date: Mon, 12 Dec 2016 22:29:44 -0700 Cc: Warner Losh , Ed Schouten , FreeBSD Hackers Message-Id: References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> <20161213052759.GP61036@server.rulingia.com> To: Peter Jeremy X-Mailer: Apple Mail (2.3124) X-Mailman-Approved-At: Tue, 13 Dec 2016 12:12:31 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 05:29:48 -0000 --Apple-Mail=_15F5C561-94BD-43D4-A6BE-9BDA98F3F57C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Dec 12, 2016, at 10:27 PM, Peter Jeremy wrote: >=20 > On 2016-Dec-12 17:57:07 -0700, Warner Losh wrote: >> On Mon, Dec 12, 2016 at 6:34 AM, Ed Schouten wrote: >>> $ time ./prometheus_sysctl_exporter | wc -l >>> 0.00 real 0.00 user 0.00 sys >>> 3162 >>>=20 >>> If our sysctl tree would get substantially larger at some point, we >>> should of course revisit this. >>=20 >> That sounds cool. I'd love to see what you came up with to get those = results. >=20 > I think that's impressive as well. >=20 >> While we're here... there's one thing I'd love to see from a >> hypothetical /dev/sysctl... When a sysctl value changes (or some >> subset), I'd like it to appear on a read-channel. That way, I could >> monitor temperature w/o needing to constantly poll for changes, for >> example. >=20 > That sounds like a job for kqueue(2), though it would need to be = plumbed > through. The biggest problem I see is that nothing in the sysctl = mechanism > has any way of reporting that a value has changed and so consumers of = that > data should re-check. This means that you'd probably just wind up = polling > in the kernel. It does if you had a read-channel :) > Note that Prometheus is designed around regularly polling the systems = it's > monitoring so this sort of functionality wouldn't benefit it. True. Warner --Apple-Mail=_15F5C561-94BD-43D4-A6BE-9BDA98F3F57C Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJYT4dJAAoJEGwc0Sh9sBEAkpMQAOBhtysDd5i/F68ZAexjHOHb CTeFvurGrJiLxygoJKHavuT5m+Onf8Oz4/zAFokw2Tcxe1wBtBlR6buAZ7Wqfo6S T6rmVPWP5U26F+mDWB57taqHPg5Ja8AfVvFTqSQi5wDGm0MvhhaH4w8MikgFOJ/l B2AlwJqFE7v/RLexpH7q57glPJY5Jj06OpDSIP2c4LaS+/8Mb2qZzoEHs8rNrBtk J0a36M1yAzp7sS7gO7zXJ0mH94xOojCl8p/AI+hQZV7bedk08YMQoetC+ZTIb/3n bA8nr1d8uxUJ8PSRGZX+FkuBefMb+Cc63VkMH9TvARf70ee6nSQYa1PVWofhssy9 7tkArgbKtQrg5pQEVmC8BOnHj28H7Ae3FEu8Esst945UTJa+pb3ZGustGdj9zwcL OK4EfGVTGN4EXPgqw6Ym/HPp28R+uz5RItPlpxannqaGTdJGvE0WMihUB7TAPAi2 IStamgs2wPY7BCZ5ixZQLbd5c0vF2jKoNDWIiGWnMqF98boSX+s5l0Gk+g6MDE8W lTA+lY0Je+XZFBKsFbEoUDH9pKTUUWKRbc1PRY4AgmOA6gRC2qD6dSHJmfXCP5yT XPlZN6q99TL8fP1pLgsl1pk1CjsD+BZq6D+2AE+tvGU/xAXKRVcUEAZv0SO9Sdrt Wy+lHinIFFRsbjENICeA =0KpV -----END PGP SIGNATURE----- --Apple-Mail=_15F5C561-94BD-43D4-A6BE-9BDA98F3F57C-- From owner-freebsd-hackers@freebsd.org Tue Dec 13 05:49:25 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7F1FBC75E25 for ; Tue, 13 Dec 2016 05:49:25 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-io0-x22d.google.com (mail-io0-x22d.google.com [IPv6:2607:f8b0:4001:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 45F381FA0 for ; Tue, 13 Dec 2016 05:49:25 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-io0-x22d.google.com with SMTP id d9so207590667ioe.0 for ; Mon, 12 Dec 2016 21:49:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=lB8WL7iF65M9oDdt9lkxcNUOhaNeSMh7qH+T/479bOc=; b=Co9SKTmfZ2ZnP65/C3Gx6SSoo2pcgtKWVC5AY9ZrbZYLOhRHwVy1B+XkR0HAraaE10 gpBkOLbIVsOuuPC/ce6AYIpoA40Or2KBGbD3pSzwPA05slAKM+jGXYbS4d5E45lmRplt 3H3vhRGDFxbUVeQDHQ5keZ6yl3x3hQks2pnnixLYFYjJxGN6v4aLgOc6ttXulY/n06b5 zG+yaRbltABOmwZpeYXhU9EzpAqJlBa4uLYtELSUeNGPFFGthVCtcA6MLbUxBFYt1cs/ kEGuRZsFGFpH2EGzm5sgHW2l2FvQUOL/dFPLEFTvjQu2XegL33sHOWzA2DrJF0Beo90l +Gmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=lB8WL7iF65M9oDdt9lkxcNUOhaNeSMh7qH+T/479bOc=; b=L3NDttuKSsBbjqI+u9DBXCcJXx48hLm9M6QrcQMS+kbYnJhzq72ZhqF4fotQ0fFWgy odh/TWLcww1hQTFgl09flzWnvuoi/NcdUbHDWxd4CT94kB/aFxbxOM236wcO9JpnM3iI 32MRscg3I0HtYDrviI31SXbTUab6UHbPjZ1J9qDmVpl8UrAoDRA1C1EulA6QfKXbn0ol mylENcneF/o85oRAbKFosT7iWgqS5RJafWQKCJkFy9JNcaHUxglcBc7cu03LzlZ+/5gH y1nP5X3hEDnyYGEHNE4NenKIZLIGOnnC6HksBwUDwVp+Dknk9547twrLU9LRya2/BGW7 1MRA== X-Gm-Message-State: AKaTC03SHcJ6qoXNhbe4Hz3VqrfQESzyCDrlB58ZLKOaDxN0gGhZDdnU6W9aeiDEQgxwjuf2SsuBZREM6DuVYQ== X-Received: by 10.36.58.85 with SMTP id m82mr1079817itm.29.1481608164605; Mon, 12 Dec 2016 21:49:24 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.150.129 with HTTP; Mon, 12 Dec 2016 21:49:23 -0800 (PST) In-Reply-To: References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> <20161213052759.GP61036@server.rulingia.com> From: Adrian Chadd Date: Mon, 12 Dec 2016 21:49:23 -0800 Message-ID: Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: Warner Losh Cc: Peter Jeremy , FreeBSD Hackers , Ed Schouten Content-Type: text/plain; charset=UTF-8 X-Mailman-Approved-At: Tue, 13 Dec 2016 12:12:50 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 05:49:25 -0000 If only we had a mechanism for these kinds of sensors to register, so the API they'd be using would be one that lead itself to edge/level triggered comparisons, versus polling. Wait, didn't we /have/ one of those up for review a few years ago, and it wasn't merged in? -adrian From owner-freebsd-hackers@freebsd.org Tue Dec 13 07:32:01 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8160BC75F20 for ; Tue, 13 Dec 2016 07:32:01 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-io0-x232.google.com (mail-io0-x232.google.com [IPv6:2607:f8b0:4001:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 454411491 for ; Tue, 13 Dec 2016 07:32:01 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-io0-x232.google.com with SMTP id d9so210621512ioe.0 for ; Mon, 12 Dec 2016 23:32:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=+1rZkESdNgqAlJQQT4ZXov8kv54l+V0nMcdvqRMpwjQ=; b=QiUxa9UzKLVwN2TYKkmGNKw62wgX+H/p039poD35LkPprbgGYFDAOBEs+M1T/xAToA ICu2TQYJqc6ZagrLKHLCX6RgaFWGYlzdyEYVHD1vyXqDpF3exOCx9FwEQdsvMfkMI7Ry b12zsJ+5WQzkYjmj3v+5MOLxPMfZBLHzzfzupAn/ZWkr6nKfcucu8HKGeyGulkyxl5kf uyu8OSe5Fso/wcYd64LOPvW9sDQSPCErNwNoaO4rLfZBFuduLZG6+BqYuaSfXXcQYMZD /NFfzhgEpXHqRggjmgPMv5cSH7xumyOSKKY5Z+qZX6YV3BNETQcQWbeVl8uIFBaOBiK4 u0ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=+1rZkESdNgqAlJQQT4ZXov8kv54l+V0nMcdvqRMpwjQ=; b=bgl5njKRKj546ZP7YHZB4TBZHCwctZdvuymNUVBTK7kExso0tL7hnjhD0xM/b1Zvn9 Gjl3en2ItIjUWU7ACxopkhF+rhLA9+L6q1vh/iwZjVSqjaYqtveDtNcURdiRR5ly68Vv H/tjK25uie4vSfHiP7zl3pmpkJ6LwrdXE/4S5Q1vqEX3NOIggch/4IiUviLr+9gyw2Lz cv4z9Bx7G9e82cTmbaGug3b0FOgfjJgr9XGdiKVtMCr8bUx6eMAfTD16boUDb3jxJpYh cRy8zP81vpilV+l+SvhbO0pzH8XvqR2ghPPnM3PNIHeY9zCRJg7uB7cBvbeGA/tmY6aG Lg/g== X-Gm-Message-State: AKaTC01tCDDPmocp7dCyWasXmCOqK4pHtZvXVUV4Sqa/VEw5Cd10cdm2wQKo2zhVFiw+D4GVNltCXWDhABlD4g== X-Received: by 10.36.58.85 with SMTP id m82mr1325063itm.29.1481614320681; Mon, 12 Dec 2016 23:32:00 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.150.129 with HTTP; Mon, 12 Dec 2016 23:32:00 -0800 (PST) In-Reply-To: References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> <20161213052759.GP61036@server.rulingia.com> From: Adrian Chadd Date: Mon, 12 Dec 2016 23:32:00 -0800 Message-ID: Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: Warner Losh Cc: Peter Jeremy , FreeBSD Hackers , Ed Schouten Content-Type: text/plain; charset=UTF-8 X-Mailman-Approved-At: Tue, 13 Dec 2016 12:13:15 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 07:32:01 -0000 On 12 December 2016 at 22:03, Warner Losh wrote: > On Mon, Dec 12, 2016 at 10:49 PM, Adrian Chadd wrote: >> If only we had a mechanism for these kinds of sensors to register, so >> the API they'd be using would be one that lead itself to edge/level >> triggered comparisons, versus polling. >> >> Wait, didn't we /have/ one of those up for review a few years ago, and >> it wasn't merged in? > > Didn't we have people willing to work with the folks that put this up > when to address the defects in the review? Oh, wait, they never wanted > to work to fix the problems, even with many offers of help to do so. > > My offer still stands, btw. Heh, this wasn't a troll? :) I should've been clearer! Does anyone remember where the ye olde framework was? -a From owner-freebsd-hackers@freebsd.org Tue Dec 13 12:41:16 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D7BDC74441 for ; Tue, 13 Dec 2016 12:41:16 +0000 (UTC) (envelope-from j.deboynepollard-newsgroups@ntlworld.com) Received: from know-smtprelay-omc-8.server.virginmedia.net (know-smtprelay-omc-8.server.virginmedia.net [80.0.253.72]) by mx1.freebsd.org (Postfix) with ESMTP id EF0671EAF for ; Tue, 13 Dec 2016 12:41:15 +0000 (UTC) (envelope-from j.deboynepollard-newsgroups@ntlworld.com) Received: from [192.168.1.100] ([86.10.211.13]) by know-smtprelay-8-imp with bizsmtp id KQg41u00G0HtmFq01Qg5AE; Tue, 13 Dec 2016 12:40:05 +0000 X-Originating-IP: [86.10.211.13] X-Spam: 0 X-Authority: v=2.1 cv=N5HUbDJB c=1 sm=1 tr=0 a=SB7hr1IvJSWWr45F2gQiKw==:117 a=SB7hr1IvJSWWr45F2gQiKw==:17 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10 a=s5jvgZ67dGcA:10 a=N659UExz7-8A:10 a=hWyk01RRhHyrYV-1E6MA:9 a=pILNOxqGKmIA:10 Subject: Re: NOTE_TRACK, EVFILT_PROC, kqueue, and subreapers References: <20161201172846.GP3428@protected.rcdrun.com> <20161201124118.46778e2b@mydesk.domain.cxm> <20161201174837.GR3428@protected.rcdrun.com> <20161201125438.15230317@mydesk.domain.cxm> <20161206104020.6b2ebb30@eto-mona.office.smartweb.sk> <20161206102637.1ddd152a@mydesk.domain.cxm> <20161207155638.4b2dd629@eto-mona.office.smartweb.sk> <630ace89-e29b-d0d3-9f15-110d8dc3de08@NTLWorld.com> <20161208132842.5d7940bd@eto-mona.office.smartweb.sk> <20161211124637.GA17315@stack.nl> From: Jonathan de Boyne Pollard To: FreeBSD Hackers , supervision@list.skarnet.org Message-ID: <67b4c42e-4e6b-ba64-b8bb-83a8a5dcfba7@NTLWorld.com> Date: Tue, 13 Dec 2016 12:39:47 +0000 User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20161211124637.GA17315@stack.nl> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 12:41:16 -0000 Jilles Tjoelker: > This should probably be fixed. There's another more insidious bug hiding inside kevent() somewhere that causes a kernel abend complaining about a sleeping thread holding a non-sleepable lock. One needs to make fairly heavy use of kevent() in order to trigger it, I believe, as I have only seen it so far on systems that do. From owner-freebsd-hackers@freebsd.org Tue Dec 13 15:08:47 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D13ECC7575D for ; Tue, 13 Dec 2016 15:08:47 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: from mail-yb0-x22b.google.com (mail-yb0-x22b.google.com [IPv6:2607:f8b0:4002:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 88C9D254 for ; Tue, 13 Dec 2016 15:08:47 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: by mail-yb0-x22b.google.com with SMTP id v78so16563260ybe.3 for ; Tue, 13 Dec 2016 07:08:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuxi-nl.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=n7tt7KZiTMC1Kh6VmsNQaAVeMD8L+4ZZiuUACuPQzDE=; b=icMa628J+X32+mTyXtUNCEERB3dTc92q5+JBSF64EZgqQhpFVZCgZpWgsgvhZm8+MQ Aym7vdj4+xd3HNQcg/f/QMqiA4p42v5ayvM4SaT6Es8KeuH9djmQzkypw6D9BL7uvupu Agb0XyseH0oxGqTTvuNhDM701GRbH5SeXpV+h4b0V8xEf2PbJLJT6ci8jScGwOIpvPQ0 /O8qoybCZR96FoBSyMLbm4fC07lAgnjlyBTCPQ21QULl+RiQykpOW+3Gxm3jkULDGDkh a7z6d5p6dHNWviPFTe15waARC9kVD2bUM3JdDOb+Vi2XQfV2oQt/aX2DNkYpDVhMHtUy Sl+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=n7tt7KZiTMC1Kh6VmsNQaAVeMD8L+4ZZiuUACuPQzDE=; b=Op01YDjy/v+qCLSW+O2xkffOb7nMPoBCLalZ8CoEb4bORujy8wy4cSCNjxfcJ0mePu YpfUaYtU8PNwNSBKjsc58CuWNHaTi7KZl1E2wv9e8WcWejbzcb0HcFQs6EiL/rmYCAMD UKJKRvl7ZWrUMGHhtRrTVKi4pFOaWGNQ7opp3m0owQVefe0CUyKl5xCkNO8X/hikkufd hzqlHRb/smmggDqujpdele9LIJnshSiPIno0Zt93dH3daSI/8QoNxOzfXlaP3FWw/UpU yHkiHgrZpbHjYdR4E7J0Mehi1KgKeCnjSdmVjBg89Yxzojy24Jfx43QCI64a+1yR4+qa f2KA== X-Gm-Message-State: AKaTC03DxDiUNJ2hLAcs57I9v15ChYd6e084jNWHWeD6F8ZfdSBTqzA5bC+h1oLtxjG0snhQvTpOuQEjFVzICg== X-Received: by 10.37.27.215 with SMTP id b206mr4921402ybb.24.1481641726491; Tue, 13 Dec 2016 07:08:46 -0800 (PST) MIME-Version: 1.0 Received: by 10.129.0.212 with HTTP; Tue, 13 Dec 2016 07:08:16 -0800 (PST) In-Reply-To: References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> From: Ed Schouten Date: Tue, 13 Dec 2016 16:08:16 +0100 Message-ID: Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: Warner Losh Cc: Jonathan Anderson , FreeBSD Hackers , Allan Jude Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 15:08:47 -0000 Hi Warner, 2016-12-13 1:57 GMT+01:00 Warner Losh : > While we're here... there's one thing I'd love to see from a > hypothetical /dev/sysctl... When a sysctl value changes (or some > subset), I'd like it to appear on a read-channel. That way, I could > monitor temperature w/o needing to constantly poll for changes, for > example. Yes, that would be nice to have. Unfortunately, Prometheus is mostly pull-based, so I guess such a feature wouldn't be needed in this case. Anyway, I just finished implementing support for attaching labels to sysctl nodes: https://reviews.freebsd.org/D8775 This patch is slightly different from what I had envisioned. For example, to translate this sysctl: hw.acpi.thermal.tz1.temperature to: sysctl_hw_acpi_thermal_temperature_celcius{thermal_zone="tz1"} my idea was to attach a "thermal_zone" label to the hw.acpi.thermal node, indicating that the names of its children should be interpreted a value for that label. Unfortunately, hw.acpi.thermal also has some children that should not have that label attached: hw.acpi.thermal.user_override hw.acpi.thermal.polling_rate hw.acpi.thermal.min_runtime That's why I went for an approach where the label is attached to hw.acpi.thermal.tz1, so that labelling can be controlled more granularly. Note that this change does not affect existing use of sysctl at all. It's merely the case that they now have additional metadata that is only used by the Prometheus exporter. -- Ed Schouten Nuxi, 's-Hertogenbosch, the Netherlands KvK-nr.: 62051717 From owner-freebsd-hackers@freebsd.org Tue Dec 13 16:16:58 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EAB00C75F91 for ; Tue, 13 Dec 2016 16:16:58 +0000 (UTC) (envelope-from mark@influxdb.com) Received: from mail-yb0-x22c.google.com (mail-yb0-x22c.google.com [IPv6:2607:f8b0:4002:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AD7B7751 for ; Tue, 13 Dec 2016 16:16:58 +0000 (UTC) (envelope-from mark@influxdb.com) Received: by mail-yb0-x22c.google.com with SMTP id d128so18334292ybh.2 for ; Tue, 13 Dec 2016 08:16:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=influxdb-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=qnI1G4I7yKPtNuxn3VHJq55UwJIDhHopNf2vy2InBDw=; b=xgs6WDhG0V9tFbOzrf1i2cC6fYMtcbWt3ZvIN9sZCETNmkMVRCPbbqfeF7FywP3Tf/ iQnC1KZF7+TWWemnu7+OfrUDz2AMw3AavNFEPOFB1MCI1YjpqoU7+0gaugaDOdddO8Kw CtzjlVulZ+4vVeGEUYJOxsmg2rlakYd6fmaQL1IY0rz3ImN33Uglkh9sNtG3zAAnDvPD zRbI6Sv0vDmgzOhq6cayqpLdrmFPl110i79ZZT7a3ytZ/UvEoffKj/tm3iw/EayfQHay qWqhd95YLdmSy/svD81wkaT1LHMC7NPf5zLMlPM9lrixC9tCTlDnyYnpc9aOUBlP2BXr nbRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=qnI1G4I7yKPtNuxn3VHJq55UwJIDhHopNf2vy2InBDw=; b=FE0WsaQUUFcTIiVakXvSsGLCyF8NbeeFFUnfmbQ952is+7c+4H/wZpuK/jhWuRIH6X 6PBZX3o+z/NfuH2TvKGcKTX39yPpiRlg6tsmH5IyZCETojuVZIq2vgQA0hDx2yK7VuKF yTj/YBTekQYRFh9U/9tNNUnSYbPOOq140+rS5Lt0TK94XsDhoCJ0BId4jX4WVpt6BqY/ 6raCRyggfuQDI4P/RwizMQ72EGEaNvGEjzDiCkeQ+FJVV5DVRheR5Ac7sCBo5X77GVZe P7U54lre7OnM1bvyqOzdd3wRS+7H6NSQkpPiIbFanBc7HOfEZI2eFV3wh2lxDux/5wAE H78A== X-Gm-Message-State: AKaTC02FIKPsuOsNz5aOeAza+Dvh7vEDJQNQE5R4VPFtKnwdiz6qDDN4WzFrdgdCfSUJ/2w9ONJwbQdasulqSNgH X-Received: by 10.37.172.216 with SMTP id x24mr5195694ybd.151.1481645817723; Tue, 13 Dec 2016 08:16:57 -0800 (PST) MIME-Version: 1.0 Received: by 10.129.153.140 with HTTP; Tue, 13 Dec 2016 08:16:57 -0800 (PST) In-Reply-To: References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> From: Mark Rushakoff Date: Tue, 13 Dec 2016 08:16:57 -0800 Message-ID: Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: Ed Schouten Cc: Jonathan Anderson , FreeBSD Hackers , Allan Jude Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 16:16:59 -0000 In Prometheus' node_exporter, it looks like they're calling sysctl to grab just CPU or just memory, etc., metrics. On the InfluxDB side of things, I'm not seeing any Telegraf plugin to monitor sysctl in a generic way, although you could probably emit InfluxDB line protocol pretty easily from the exporter tool or something similar to that. My impression, from a monitoring perspective, is that there isn't much demand to gather _everything_ in sysctl in a single call, although categorized subsets might be appealing. On Mon, Dec 12, 2016 at 5:34 AM, Ed Schouten wrote: > Hi Jonathan, > > 2016-12-11 21:12 GMT+01:00 Jonathan Anderson : > > Rather than doing lots of system calls to > > enumerate clearly-related sysctl values, it would be very cool to do one > > system call and then inspect the resulting data structure in memory. > > I'm not sure it's worth tackling this problem at this time. My current > version of the exporter is already pretty fast: > > $ time ./prometheus_sysctl_exporter | wc -l > 0.00 real 0.00 user 0.00 sys > 3162 > > If our sysctl tree would get substantially larger at some point, we > should of course revisit this. > > -- > Ed Schouten > Nuxi, 's-Hertogenbosch, the Netherlands > KvK-nr.: 62051717 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Tue Dec 13 16:24:28 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E20FC7643E for ; Tue, 13 Dec 2016 16:24:28 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9399C10AB for ; Tue, 13 Dec 2016 16:24:27 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x22c.google.com with SMTP id f82so126582331wmf.1 for ; Tue, 13 Dec 2016 08:24:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=Si1qoKmC2kTrk5liix1akJMS+icz+aaGTDv5lRer70Y=; b=uE7gSp810jKZnxkfZ5U3JFbF48Hr52DUYfBvXMVQ3Otf5OOZFqhc6M/iPgHpmJUwLJ h7gNAVEb3+f26et3+AdM5QWLettXj3x54CGVtzybbi9R9dkJ31dMWmAM2CuQRwhQEiVJ eCwg9eh9KMNcirdoTc/v9Hd7UHR48tWA9dZ3NA8LKP7gvJporoIH4xJ2QwHP8RxOj8n2 8iIHiqoEoNFCWxxTyO+96hM3we4MjIaFntNWfATZ5ahsuWY4HBIEpl9eS8TgkiGfbUwu KjeURuZP6Dm44O2RyhWvGn47yDYb/S8pi0JGnZ5xtrMinpjwgH2YYOzp62tOmNFvu2g5 NfBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=Si1qoKmC2kTrk5liix1akJMS+icz+aaGTDv5lRer70Y=; b=YLAXhcezRgrw8enAIsAEiGlT0DRqnq/203jmXVgGHwkT9X5qw5WzPv6yS7wFYNm971 wJV7ul2gQZlCuGAssHQkNGWLYGbWHMmlRP4ULdGW2XSQwRNrcK4TNXtqwv/Y8pd3SBx4 k0ubm9nCcYNShc2QYXb3CFwU4BSI6ajyobpTZQdFmX0yojZHnMa8/iyi2K+YwVKnIK23 wGQM4myZfVzRmwl1Gzskzz0uFg6WYXCSQVfGV2/w05SGqUOY3H90KKXjd/BncbGq9XjH M2z+PPR9j9M4tThnWdmEOYyN4wCWqvNlb2+dVHaAr2aXrryMFcflRXR2Gv2oKxoLLdBl AfRg== X-Gm-Message-State: AKaTC00Di/hXNLyDYLUf1NGkEuY13RVd+S4+LFCqc23UBaGCjcyeH/ARUw0u1t4YXdhzbsD6 X-Received: by 10.28.138.137 with SMTP id m131mr3382515wmd.118.1481646265584; Tue, 13 Dec 2016 08:24:25 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id w8sm3339057wmw.4.2016.12.13.08.24.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Dec 2016 08:24:24 -0800 (PST) Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: freebsd-hackers@freebsd.org References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> From: Steven Hartland Message-ID: Date: Tue, 13 Dec 2016 16:25:02 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 16:24:28 -0000 Gathering every sysctl would case issues as some are very heavy and if called regularly would cause noticeable stalls. On 13/12/2016 16:16, Mark Rushakoff wrote: > In Prometheus' node_exporter, it looks like they're calling sysctl to grab > just > CPU or just memory, etc., metrics. > > On the InfluxDB side of things, I'm not seeing any Telegraf plugin to > monitor > sysctl in a generic way, although you could probably emit InfluxDB line > protocol pretty easily from the exporter tool or something similar to that. > > My impression, from a monitoring perspective, is that there isn't much > demand > to gather _everything_ in sysctl in a single call, although categorized > subsets > might be appealing. > > > On Mon, Dec 12, 2016 at 5:34 AM, Ed Schouten wrote: > >> Hi Jonathan, >> >> 2016-12-11 21:12 GMT+01:00 Jonathan Anderson : >>> Rather than doing lots of system calls to >>> enumerate clearly-related sysctl values, it would be very cool to do one >>> system call and then inspect the resulting data structure in memory. >> I'm not sure it's worth tackling this problem at this time. My current >> version of the exporter is already pretty fast: >> >> $ time ./prometheus_sysctl_exporter | wc -l >> 0.00 real 0.00 user 0.00 sys >> 3162 >> >> If our sysctl tree would get substantially larger at some point, we >> should of course revisit this. >> >> -- >> Ed Schouten >> Nuxi, 's-Hertogenbosch, the Netherlands >> KvK-nr.: 62051717 >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >> > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Tue Dec 13 16:31:36 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04C41C76939 for ; Tue, 13 Dec 2016 16:31:36 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B9DB7192D; Tue, 13 Dec 2016 16:31:35 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1cGpzC-0001HK-MT; Tue, 13 Dec 2016 19:31:26 +0300 Date: Tue, 13 Dec 2016 19:31:26 +0300 From: Slawa Olhovchenkov To: Mark Rushakoff Cc: Ed Schouten , FreeBSD Hackers , Allan Jude , Jonathan Anderson Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems Message-ID: <20161213163126.GA98176@zxy.spb.ru> References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 16:31:36 -0000 On Tue, Dec 13, 2016 at 08:16:57AM -0800, Mark Rushakoff wrote: > In Prometheus' node_exporter, it looks like they're calling sysctl to grab > just > CPU or just memory, etc., metrics. > > On the InfluxDB side of things, I'm not seeing any Telegraf plugin to > monitor > sysctl in a generic way, although you could probably emit InfluxDB line > protocol pretty easily from the exporter tool or something similar to that. > > My impression, from a monitoring perspective, is that there isn't much > demand > to gather _everything_ in sysctl in a single call, although categorized > subsets > might be appealing. Many sysctl nodes have a) opaque binary values, w/ version depended formats b) opaque binary values w/ size more then values c) very large size: sysctl -b kern.msgbuf|wc 10200 55998 1048505 Also, InfluxDB is hipster shit and for many cases need to preprocess collected values, because imposible to calculate on Influx side. From owner-freebsd-hackers@freebsd.org Tue Dec 13 17:41:39 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2FDBC76A4E for ; Tue, 13 Dec 2016 17:41:39 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [88.99.82.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 79F0E19D6; Tue, 13 Dec 2016 17:41:38 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 5CEAE1FE157; Tue, 13 Dec 2016 18:34:37 +0100 (CET) Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: Ed Schouten , Warner Losh References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> Cc: FreeBSD Hackers , Allan Jude , Jonathan Anderson From: Hans Petter Selasky Message-ID: Date: Tue, 13 Dec 2016 18:34:09 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 17:41:39 -0000 On 12/13/16 16:08, Ed Schouten wrote: > Hi Warner, > > 2016-12-13 1:57 GMT+01:00 Warner Losh : >> > While we're here... there's one thing I'd love to see from a >> > hypothetical /dev/sysctl... When a sysctl value changes (or some >> > subset), I'd like it to appear on a read-channel. That way, I could >> > monitor temperature w/o needing to constantly poll for changes, for >> > example. > Yes, that would be nice to have. Unfortunately, Prometheus is mostly > pull-based, so I guess such a feature wouldn't be needed in this case. Maybe you could make something with libcuse() ? --HPS From owner-freebsd-hackers@freebsd.org Tue Dec 13 18:12:33 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F1593C76924 for ; Tue, 13 Dec 2016 18:12:33 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: from mail-yb0-x230.google.com (mail-yb0-x230.google.com [IPv6:2607:f8b0:4002:c09::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A2E261635 for ; Tue, 13 Dec 2016 18:12:33 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: by mail-yb0-x230.google.com with SMTP id d59so21302771ybi.1 for ; Tue, 13 Dec 2016 10:12:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuxi-nl.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=+bqc79gn+qeu+Ai4sdd99k/0sBmRnZ5+8v2Tufi64VA=; b=B8QFkKtAcw/Dm1Z/qx0HGWlYImuk7+XaFxyzk7N6wmdtGfvPDPolp3OhN+KQdfv+AM e7eDeRjhvshKhjF5L4lbRvYFhgZcOyQ9WZDAooIRlHNcsbS+Xa8ZAjwjpjNZQLcpvZgy TbblPOXvHl1eIncnFh8NVq4zgN5c2zbprKOWlSwYQ46f56JHPikaqogoT6pIid6UsLKX +ElWJ8sD34iEF/LRl1wYgpYAlwAhCg3XZ5W3baRqumvaAZ65KmYuDuTIkXAZhA8vzrIh cN4oMW0BVRth+Hjh3Srw+/MHRJY+EX0ylrv94J9NxnifBqLcA5YIPrIZMCz28zeWFGZe SRjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=+bqc79gn+qeu+Ai4sdd99k/0sBmRnZ5+8v2Tufi64VA=; b=FRQGgLda1XFq16ouEEYkVhUkBAPDxNgOzxgC13np6ngpXcZ+Or4Lw8nB3s/cg2OzKj QosoTWxSqAwi8M5KrLGs1eiE2xbxPTDB351FNt7i7RYIE1iEru3aWir2AKOFkCJNnpXk t1kLvG/AbSfvaEGewboVgUxbfjv1JoFKmfT25EhUSrYROzgfAOo3rvoh5We9v/Axqyyy OOhhMrwBh7S8a1pLKNDg604FWLWSHFAF0WG+J4RWboTG7eNTdYckjPQbd/Wh1s4foUa1 w1vxd+Fkp7Si1W54noi6+tAgVAXgcmSnGsfFMFQHNF2PoizbRXhzwxByLAFQT24q4ols QCPg== X-Gm-Message-State: AKaTC01nim0KcrNOEDOH2p7VU/RHyAAceR+65lCsKgyMyFk/dO/Dcbr2uPFrt4nvU7blpQyt98pUxNgqTXoKEw== X-Received: by 10.37.210.197 with SMTP id j188mr50362887ybg.168.1481652752828; Tue, 13 Dec 2016 10:12:32 -0800 (PST) MIME-Version: 1.0 Received: by 10.129.0.212 with HTTP; Tue, 13 Dec 2016 10:12:02 -0800 (PST) In-Reply-To: References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> From: Ed Schouten Date: Tue, 13 Dec 2016 19:12:02 +0100 Message-ID: Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: Mark Rushakoff Cc: Jonathan Anderson , FreeBSD Hackers , Allan Jude Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 18:12:34 -0000 2016-12-13 17:16 GMT+01:00 Mark Rushakoff : > My impression, from a monitoring perspective, is that there isn't much > demand to gather _everything_ in sysctl in a single call, although categorized > subsets might be appealing. Exactly. Right now my exporter simply dumps everything, but my intention it so that you can also pass in a list of prefixes on the command line, like this: $ prometheus_sysctl_exporter hw.acpi.thermal kern.openfiles kern.maxfiles This will make it export only ACPI thermal metrics and selected values from kern. Still, the advantage of having something like this is that it significantly lowers the barrier for accessing such metrics. Right now we first need to add it to FreeBSD and then hope that someone out there takes the effort to write a plugin. Exporting all metrics is also not that bad if that's really what you're interested in. On my simple system the exporter only returns about ~1500 metrics, as it can only extract sysctls with integer values. From personal experience I know that Prometheus can handle that easily, especially when it happens on the same physical network, where bandwidth is not an issue. Discarding uninteresting data can also be performed when federating. -- Ed Schouten Nuxi, 's-Hertogenbosch, the Netherlands KvK-nr.: 62051717 From owner-freebsd-hackers@freebsd.org Tue Dec 13 18:17:11 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2F9FFC76A5A for ; Tue, 13 Dec 2016 18:17:11 +0000 (UTC) (envelope-from hrant@dadivanyan.net) Received: from pandora.amnic.net (pandora.amnic.net [IPv6:2001:67c:21c:a610::11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B80EC194F; Tue, 13 Dec 2016 18:17:10 +0000 (UTC) (envelope-from hrant@dadivanyan.net) Received: from ran by pandora.amnic.net with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1cGrdS-00056Q-FR; Tue, 13 Dec 2016 22:17:06 +0400 Subject: Re: system time instability In-Reply-To: <20161213103703.GR54029@kib.kiev.ua> To: Konstantin Belousov Date: Tue, 13 Dec 2016 22:17:06 +0400 (AMT) From: Hrant Dadivanyan CC: Ian Lepore , freebsd-hackers@freebsd.org X-PGP: https://amnic.net/pgpkeys/hrant.asc X-NCC-RegID: am.isoc X-Mailer: ELM [version 2.4ME+ PL127 (25)] MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="ELM1481653026-17779-0_" Content-Transfer-Encoding: 7bit Message-Id: X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 18:17:11 -0000 --ELM1481653026-17779-0_ Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="UTF-8" > On Mon, Dec 12, 2016 at 11:33:59PM +0400, Hrant Dadivanyan wrote: > > > On Mon, Dec 12, 2016 at 11:04:08PM +0400, Hrant Dadivanyan wrote: > > > > Now, when you ask, I start to suspect PPS delivery to uart again - cable > > > > and amplifier, but can't understand how the 100ppm error fits into that. > > > > > > If you disable PPS sync in ntp config, does the machine keep time adequately ? > > > > > > > Thanks for reminding - yes, I've tried this as well, the issue persists. > > So uart shouldn't be in charge. > > This statement seems to be wrong, look below. > > > There might be relatively long pauses when system management mode handlers > > > do something in response to hw events. E.g. if you have USB emulation of > > > AT keyboard enabled in BIOS, try to disable that. And update the BIOS. > > > > The USB is switched off in the BIOS. I've removed all changes in sysctl.conf > > and nice flag from ntpd, recompiled kernel as following: > > include GENERIC > > options PPS_SYNC > > device pf > > device pflog > > and started over. Dmesg is attached. > > > Please show verbose dmesg. > I've updated BIOS to the latest one. Verbose dmesg is attached. > > CPU: Intel(R) Core(TM)2 Duo CPU E4500 @ 2.20GHz (2194.55-MHz K8-class CPU) > > This is relatively old CPU which is known to have some (minor) issues with > interaction between power saving and cores. Try the following OS config: > disable deep C states, allow only C1 (there might be some tweaks in BIOS, > if possible, disable the Cn, n > 1, there too); Have never touched Cx states on servers, it was disabled in BIOS and sysctl shows C1 as lowest. Now I've enabled it in BIOS, but didn't touch in OS: dev.cpu.1.cx_usage: 100.00% last 31000us dev.cpu.1.cx_lowest: C1 dev.cpu.1.cx_supported: C1/1/0 dev.cpu.0.cx_usage: 100.00% last 5569us dev.cpu.0.cx_lowest: C1 dev.cpu.0.cx_supported: C1/1/0 dev.cpu.0.freq_levels: 2200/35000 2000/31000 1800/27000 1600/23000 1400/19000 1200/16000 dev.cpu.0.freq: 2200 Is this correct ? > use LAPIC for event timer (not HPET); Have disabled HPET in BIOS: kern.eventtimer.periodic: 0 kern.eventtimer.timer: LAPIC kern.eventtimer.idletick: 0 kern.eventtimer.singlemul: 2 kern.eventtimer.choice: LAPIC(400) i8254(100) RTC(0) kern.eventtimer.et.i8254.quality: 100 kern.eventtimer.et.i8254.frequency: 1193182 kern.eventtimer.et.i8254.flags: 1 kern.eventtimer.et.RTC.quality: 0 kern.eventtimer.et.RTC.frequency: 32768 kern.eventtimer.et.RTC.flags: 17 kern.eventtimer.et.LAPIC.quality: 400 kern.eventtimer.et.LAPIC.frequency: 99751860 kern.eventtimer.et.LAPIC.flags: 15 > re-check that you use RDTSC for the timecounter; kern.timecounter.tsc_shift: 1 kern.timecounter.smp_tsc_adjust: 0 kern.timecounter.smp_tsc: 1 kern.timecounter.invariant_tsc: 1 kern.timecounter.fast_gettime: 1 kern.timecounter.tick: 1 kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) i8254(0) dummy(-1000000) kern.timecounter.hardware: TSC-low kern.timecounter.alloweddeviation: 5 kern.timecounter.stepwarnings: 0 kern.timecounter.tc.TSC-low.quality: 1000 kern.timecounter.tc.TSC-low.frequency: 1097249250 kern.timecounter.tc.TSC-low.counter: 1335765171 kern.timecounter.tc.TSC-low.mask: 4294967295 kern.timecounter.tc.ACPI-fast.quality: 900 kern.timecounter.tc.ACPI-fast.frequency: 3579545 kern.timecounter.tc.ACPI-fast.counter: 6343821 kern.timecounter.tc.ACPI-fast.mask: 16777215 kern.timecounter.tc.i8254.quality: 0 kern.timecounter.tc.i8254.frequency: 1193182 kern.timecounter.tc.i8254.counter: 2701 kern.timecounter.tc.i8254.mask: 65535 > do not enable powerd. > Never did on servers. > You might also try the stable/11 kernel, which has more changes WRT C-states > handling and PPS/ntp locking. > The server did run for almost a day without PPS and looks stable. I start to believe, to my shame, that I did a mistake when testing this previously. Then the whole post is wrong and cable seems to be most suspected part again. Even now it's hard to understand this wrong behaviour, but anyway ... Just replaced the cable with shielded one where each pair has separate shield, used dedicated pair for PPS and ground; grounded the shields. Thank you Konstantin, thank you Ian ! Hrant -- Hrant Dadivanyan (aka Ran d'Adi) hrant(at)dadivanyan.net /* "Feci quod potui, faciant meliora potentes." */ ran(at)psg.com --ELM1481653026-17779-0_ Content-Transfer-Encoding: 7bit Content-Type: text/plain Content-Disposition: attachment; filename="dmesg.v" Content-Description: Table 'FACP' at 0x7f68ad08 Table 'TCPA' at 0x7f68ad8c Table 'MCFG' at 0x7f68adbe Table 'APIC' at 0x7f68adfa APIC: Found table at 0x7f68adfa APIC: Using the MADT enumerator. MADT: Found CPU APIC ID 0 ACPI ID 0: enabled SMP: Added CPU 0 (AP) MADT: Found CPU APIC ID 1 ACPI ID 1: enabled SMP: Added CPU 1 (AP) Copyright (c) 1992-2016 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.3-RELEASE-p14 #0 r309851: Mon Dec 12 19:38:52 AMT 2016 ran@ntp.amnic.net:/usr/obj/usr/src/sys/NTP amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff819f3000. Calibrating TSC clock ... TSC clock: 2194547245 Hz CPU: Intel(R) Core(TM)2 Duo CPU E4500 @ 2.20GHz (2194.55-MHz K8-class CPU) Origin="GenuineIntel" Id=0x6fd Family=0x6 Model=0xf Stepping=13 Features=0xbfebfbff Features2=0xe39d AMD Features=0x20100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics Instruction TLB: 2M pages, 4-way, 8 entries or 4M pages, 4-way, 4 entries Instruction TLB: 4 KB Pages, 4-way set associative, 128 entries 64-Byte prefetching Data TLB0: 4 KByte pages, 4-way associative, 16 entries Data TLB0: 4 MByte pages, 4-way set associative, 16 entries 2nd-level cache: 2-MB, 8-way set associative, 64-byte line size 1st-level instruction cache: 32 KB, 8-way set associative, 64 byte line size Data TLB1: 4 KByte pages, 4-way associative, 256 entries 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size L2 cache: 2048 kbytes, 8-way associative, 64 bytes/line real memory = 2147483648 (2048 MB) Physical memory chunk(s): 0x0000000000010000 - 0x0000000000091fff, 532480 bytes (130 pages) 0x0000000000100000 - 0x00000000001fffff, 1048576 bytes (256 pages) 0x0000000001a35000 - 0x000000007c245fff, 2055278592 bytes (501777 pages) avail memory = 2040201216 (1945 MB) Event timer "LAPIC" quality 400 LAPIC: ipi_wait() us multiplier 55 (r 39750084 tsc 2194547245) ACPI APIC Table: INTR: Adding local APIC 1 as a target FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 APIC: CPU 0 has ACPI ID 0 APIC: CPU 1 has ACPI ID 1 x86bios: IVT 0x000000-0x0004ff at 0xfffff80000000000 x86bios: SSEG 0x090000-0x090fff at 0xfffffe007b5ef000 x86bios: EBDA 0x096000-0x09ffff at 0xfffff80000096000 x86bios: ROM 0x0a0000-0x0fefff at 0xfffff800000a0000 XEN: CPU 0 has VCPU ID 0 XEN: CPU 1 has VCPU ID 1 Hardware, Intel Secure Key RNG: RDRAND is not present Hardware, VIA Nehemiah Padlock RNG: VIA Padlock RNG not present Falling back to random adaptor random: initialized ULE: setup cpu 0 ULE: setup cpu 1 ACPI: RSDP 0x00000000000F7260 000014 (v00 PTLTD ) ACPI: RSDT 0x000000007F681E06 000060 (v01 PTLTD RSDT 06040000 LTP 00000000) ACPI: FACP 0x000000007F68AD08 000084 (v01 SUPRMC 06040000 PTL 00000003) ACPI: DSDT 0x000000007F68569F 005669 (v01 INTEL BR_WATER 06040000 MSFT 03000001) ACPI: FACS 0x000000007F68BFC0 000040 ACPI: TCPA 0x000000007F68AD8C 000032 (v01 SMC 06040000 PTL 00000000) ACPI: MCFG 0x000000007F68ADBE 00003C (v01 PTLTD MCFG 06040000 LTP 00000000) ACPI: APIC 0x000000007F68ADFA 000068 (v01 PTLTD ? APIC 06040000 LTP 00000000) ACPI: BOOT 0x000000007F68AE62 000028 (v01 PTLTD $SBFTBL$ 06040000 LTP 00000001) ACPI: SLIC 0x000000007F68AE8A 000176 (v01 OEMID_ OEMTABLE 06040000 LTP 00000000) ACPI: SSDT 0x000000007F6836D9 00025F (v01 PmRef Cpu0Tst 00003000 INTL 20050228) ACPI: SSDT 0x000000007F683633 0000A6 (v01 PmRef Cpu7Tst 00003000 INTL 20050228) ACPI: SSDT 0x000000007F68358D 0000A6 (v01 PmRef Cpu6Tst 00003000 INTL 20050228) ACPI: SSDT 0x000000007F6834E7 0000A6 (v01 PmRef Cpu5Tst 00003000 INTL 20050228) ACPI: SSDT 0x000000007F683441 0000A6 (v01 PmRef Cpu4Tst 00003000 INTL 20050228) ACPI: SSDT 0x000000007F68339B 0000A6 (v01 PmRef Cpu3Tst 00003000 INTL 20050228) ACPI: SSDT 0x000000007F6832F5 0000A6 (v01 PmRef Cpu2Tst 00003000 INTL 20050228) ACPI: SSDT 0x000000007F68324F 0000A6 (v01 PmRef Cpu1Tst 00003000 INTL 20050228) ACPI: SSDT 0x000000007F681E66 0013E9 (v01 PmRef CpuPm 00003000 INTL 20050228) MADT: Found IO APIC ID 2, Interrupt 0 at 0xfec00000 ioapic0: Routing external 8259A's -> intpin 0 lapic0: Routing NMI -> LINT1 lapic0: LINT1 trigger: edge lapic0: LINT1 polarity: high lapic1: Routing NMI -> LINT1 lapic1: LINT1 trigger: edge lapic1: LINT1 polarity: high MADT: Interrupt override: source 0, irq 2 ioapic0: Routing IRQ 0 -> intpin 2 MADT: Interrupt override: source 9, irq 9 ioapic0: intpin 9 trigger: level ioapic0 irqs 0-23 on motherboard cpu0 BSP: ID: 0x00000000 VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00000200 err: 0x000000f0 pmc: 0x00010400 snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024] feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25 wlan: <802.11 Link Layer> null: nfslock: pseudo-device VESA: INT 0x10 vector 0xc000:0x0014 VESA: information block 0000 56 45 53 41 00 03 00 01 00 91 01 00 00 00 40 00 0010 00 91 77 00 00 01 38 01 00 91 4a 01 00 91 73 01 0020 00 91 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0040 60 01 61 01 62 01 63 01 64 01 65 01 66 01 67 01 0050 68 01 3c 01 4d 01 5c 01 3a 01 4b 01 5a 01 07 01 0060 1a 01 1b 01 05 01 17 01 18 01 12 01 14 01 15 01 0070 01 01 03 01 11 01 ff ff 00 00 00 00 00 00 00 00 0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0100 49 6e 74 65 6c 28 72 29 42 72 6f 61 64 77 61 74 0110 65 72 2d 47 20 47 72 61 70 68 69 63 73 20 43 68 0120 69 70 20 41 63 63 65 6c 65 72 61 74 65 64 20 56 0130 47 41 20 42 49 4f 53 00 49 6e 74 65 6c 20 43 6f 0140 72 70 6f 72 61 74 69 6f 6e 00 49 6e 74 65 6c 28 0150 72 29 42 72 6f 61 64 77 61 74 65 72 2d 47 20 47 0160 72 61 70 68 69 63 73 20 43 6f 6e 74 72 6f 6c 6c 0170 65 72 00 48 61 72 64 77 61 72 65 20 56 65 72 73 0180 69 6f 6e 20 30 2e 30 00 00 00 00 00 00 00 00 00 0190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 VESA: 17 mode(s) found VESA: v3.0, 7616k memory, flags:0x1, mode table:0xfffffe007b5ff040 (91000040) VESA: Intel(r)Broadwater-G Graphics Chip Accelerated VGA BIOS VESA: Intel Corporation Intel(r)Broadwater-G Graphics Controller Hardware Version 0.0 io: VMBUS: load kbd: new array size 4 kbd1 at kbdmux0 mem: hpt27xx: RocketRAID 27xx controller driver v1.2.7 hptrr: RocketRAID 17xx/2xxx SATA controller driver v1.2 hptnr: R750/DC7280 controller driver v1.1.4 acpi0: on motherboard ACPI: All ACPI Tables successfully acquired PCIe: Memory Mapped configuration base @ 0xe0000000 ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48 cpu0: Processor \134_PR_.CPU0 (ACPI ID 0) -> APIC ID 0 cpu0: on acpi0 ACPI: Dynamic OEM Table Load: ACPI: SSDT 0xFFFFF80002532000 0002E5 (v01 PmRef Cpu0Ist 00003000 INTL 20050228) cpu0: switching to generic Cx mode cpu1: Processor \134_PR_.CPU1 (ACPI ID 1) -> APIC ID 1 cpu1: on acpi0 ACPI: Dynamic OEM Table Load: ACPI: SSDT 0xFFFFF80002531800 000276 (v01 PmRef Cpu1Ist 00003000 INTL 20050228) ACPI: Processor \134_PR_.CPU2 (ACPI ID 2) ignored ACPI: Processor \134_PR_.CPU3 (ACPI ID 3) ignored ACPI: Processor \134_PR_.CPU4 (ACPI ID 4) ignored ACPI: Processor \134_PR_.CPU5 (ACPI ID 5) ignored ACPI: Processor \134_PR_.CPU6 (ACPI ID 6) ignored ACPI: Processor \134_PR_.CPU7 (ACPI ID 7) ignored atrtc0: port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock (resolution 1000000us, adjustment 0.500000000s) ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 0 vector 49 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 ioapic0: routing intpin 2 (ISA IRQ 0) to lapic 0 vector 50 Event timer "i8254" frequency 1193182 Hz quality 100 ACPI timer: 1/1 1/1 1/1 1/1 1/1 1/1 1/1 1/1 1/1 1/1 -> 10 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pci_link0: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 3 4 5 6 7 10 11 12 14 15 Validation 0 10 N 0 3 4 5 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 5 6 7 10 11 12 14 15 pci_link1: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 3 4 5 6 7 10 11 12 14 15 Validation 0 11 N 0 3 4 5 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 5 6 7 10 11 12 14 15 pci_link2: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 10 11 12 14 15 Validation 0 255 N 0 3 4 5 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 5 6 7 10 11 12 14 15 pci_link3: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 10 11 12 14 15 Validation 0 255 N 0 3 4 5 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 5 6 7 10 11 12 14 15 pci_link4: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 10 11 12 14 15 Validation 0 255 N 0 3 4 5 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 5 6 7 10 11 12 14 15 pci_link5: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 10 11 12 14 15 Validation 0 255 N 0 3 4 5 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 5 6 7 10 11 12 14 15 pci_link6: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 10 11 12 14 15 Validation 0 255 N 0 3 4 5 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 5 6 7 10 11 12 14 15 pci_link7: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 5 6 7 10 11 12 14 15 Validation 0 255 N 0 3 4 5 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 5 6 7 10 11 12 14 15 pcib0: port 0xcf8-0xcff on acpi0 pcib0: decoding 5 range 0-0xff pcib0: decoding 4 range 0-0xcf7 pcib0: decoding 3 range 0xa0000-0xbffff pcib0: decoding 3 range 0xd4000-0xd7fff pcib0: decoding 3 range 0xd8000-0xdbfff pcib0: decoding 3 range 0xdc000-0xdffff pcib0: decoding 3 range 0xe0000-0xe3fff pcib0: decoding 3 range 0x80000000-0xdfffffff pcib0: decoding 4 range 0xd00-0xfdff pcib0: decoding 3 range 0xfed40000-0xfed44fff pci0: on pcib0 pci0: domain=0, physical bus=0 found-> vendor=0x8086, dev=0x2970, revid=0x02 domain=0, bus=0, slot=0, func=0 class=06-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0106, statreg=0x2090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x2972, revid=0x02 domain=0, bus=0, slot=2, func=0 class=03-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x0090, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 2 supports D0 D3 current D0 MSI supports 1 message map[10]: type Memory, range 32, base 0xd0000000, size 20, enabled pcib0: allocated type 3 (0xd0000000-0xd00fffff) for rid 10 of pci0:0:2:0 map[18]: type Prefetchable Memory, range 64, base 0xc0000000, size 28, enabled pcib0: allocated type 3 (0xc0000000-0xcfffffff) for rid 18 of pci0:0:2:0 map[20]: type I/O Port, range 32, base 0x3030, size 3, enabled pcib0: allocated type 4 (0x3030-0x3037) for rid 20 of pci0:0:2:0 pcib0: matched entry for 0.2.INTA pcib0: slot 2 INTA hardwired to IRQ 16 found-> vendor=0x8086, dev=0x27d0, revid=0x01 domain=0, bus=0, slot=28, func=0 class=06-04-00, hdrtype=0x01, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 2 supports D0 D3 current D0 MSI supports 1 message pcib0: matched entry for 0.28.INTA pcib0: slot 28 INTA hardwired to IRQ 16 secbus=1, subbus=1 found-> vendor=0x8086, dev=0x27e0, revid=0x01 domain=0, bus=0, slot=28, func=4 class=06-04-00, hdrtype=0x01, mfdev=1 cmdreg=0x0007, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 2 supports D0 D3 current D0 MSI supports 1 message pcib0: matched entry for 0.28.INTA pcib0: slot 28 INTA hardwired to IRQ 16 secbus=2, subbus=2 found-> vendor=0x8086, dev=0x27e2, revid=0x01 domain=0, bus=0, slot=28, func=5 class=06-04-00, hdrtype=0x01, mfdev=1 cmdreg=0x0007, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=11 powerspec 2 supports D0 D3 current D0 MSI supports 1 message pcib0: matched entry for 0.28.INTB pcib0: slot 28 INTB hardwired to IRQ 17 secbus=3, subbus=3 found-> vendor=0x8086, dev=0x244e, revid=0xe1 domain=0, bus=0, slot=30, func=0 class=06-04-01, hdrtype=0x01, mfdev=0 cmdreg=0x0107, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) secbus=4, subbus=4 found-> vendor=0x8086, dev=0x27b8, revid=0x01 domain=0, bus=0, slot=31, func=0 class=06-01-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0210, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x27c3, revid=0x01 domain=0, bus=0, slot=31, func=2 class=01-04-00, hdrtype=0x00, mfdev=0 cmdreg=0x0007, statreg=0x02b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=11 powerspec 2 supports D0 D3 current D0 MSI supports 1 message map[10]: type I/O Port, range 32, base 0x3048, size 3, enabled pcib0: allocated type 4 (0x3048-0x304f) for rid 10 of pci0:0:31:2 map[14]: type I/O Port, range 32, base 0x303c, size 2, enabled pcib0: allocated type 4 (0x303c-0x303f) for rid 14 of pci0:0:31:2 map[18]: type I/O Port, range 32, base 0x3040, size 3, enabled pcib0: allocated type 4 (0x3040-0x3047) for rid 18 of pci0:0:31:2 map[1c]: type I/O Port, range 32, base 0x3038, size 2, enabled pcib0: allocated type 4 (0x3038-0x303b) for rid 1c of pci0:0:31:2 map[20]: type I/O Port, range 32, base 0x3020, size 4, enabled pcib0: allocated type 4 (0x3020-0x302f) for rid 20 of pci0:0:31:2 map[24]: type Memory, range 32, base 0xd0500000, size 10, enabled pcib0: allocated type 3 (0xd0500000-0xd05003ff) for rid 24 of pci0:0:31:2 pcib0: matched entry for 0.31.INTB pcib0: slot 31 INTB hardwired to IRQ 17 found-> vendor=0x8086, dev=0x27da, revid=0x01 domain=0, bus=0, slot=31, func=3 class=0c-05-00, hdrtype=0x00, mfdev=0 cmdreg=0x0001, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=11 map[20]: type I/O Port, range 32, base 0x1100, size 5, enabled pcib0: allocated type 4 (0x1100-0x111f) for rid 20 of pci0:0:31:3 pcib0: matched entry for 0.31.INTB pcib0: slot 31 INTB hardwired to IRQ 17 vgapci0: port 0x3030-0x3037 mem 0xd0000000-0xd00fffff,0xc0000000-0xcfffffff irq 16 at device 2.0 on pci0 agp0: on vgapci0 pcib0: allocated type 3 (0x80000000-0x80000fff) for rid 64 of agp0 agp0: Allocated flush page phys 0x80000000 virt 0xfffff80080000000 agp0: aperture size is 256M, detected 7676k stolen memory agp0: AGP_I965_PGTBL_CTL2: 00000000 agp0: AGP_I855_GCC1: 0x30 agp0: AGP_I965_MSAC: 0x00 agp0: Mappable GTT entries: 65536 agp0: Total GTT entries: 131072 vgapci0: Boot video device pcib1: irq 16 at device 28.0 on pci0 pcib1: domain 0 pcib1: secondary bus 1 pcib1: subordinate bus 1 pci1: on pcib1 pcib1: allocated bus range (1-1) for rid 0 of pci1 pci1: domain=0, physical bus=1 pcib2: irq 16 at device 28.4 on pci0 pcib2: allocating non-ISA range 0x4000-0x40ff pcib0: allocated type 4 (0x4000-0x40ff) for rid 1c of pcib2 pcib2: allocating non-ISA range 0x4400-0x44ff pcib0: allocated type 4 (0x4400-0x44ff) for rid 1c of pcib2 pcib2: allocating non-ISA range 0x4800-0x48ff pcib0: allocated type 4 (0x4800-0x48ff) for rid 1c of pcib2 pcib2: allocating non-ISA range 0x4c00-0x4cff pcib0: allocated type 4 (0x4c00-0x4cff) for rid 1c of pcib2 pcib0: allocated type 3 (0xd0100000-0xd01fffff) for rid 20 of pcib2 pcib2: domain 0 pcib2: secondary bus 2 pcib2: subordinate bus 2 pcib2: I/O decode 0x4000-0x4fff pcib2: memory decode 0xd0100000-0xd01fffff pcib2: special decode ISA pci2: on pcib2 pcib2: allocated bus range (2-2) for rid 0 of pci2 pci2: domain=0, physical bus=2 found-> vendor=0x8086, dev=0x109a, revid=0x00 domain=0, bus=2, slot=0, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0107, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 2 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 32, base 0xd0100000, size 17, enabled pcib2: allocated memory range (0xd0100000-0xd011ffff) for rid 10 of pci0:2:0:0 map[18]: type I/O Port, range 32, base 0x4000, size 5, enabled pcib2: allocated I/O port range (0x4000-0x401f) for rid 18 of pci0:2:0:0 pcib2: matched entry for 2.0.INTA pcib2: slot 0 INTA hardwired to IRQ 16 em0: port 0x4000-0x401f mem 0xd0100000-0xd011ffff irq 16 at device 0.0 on pci2 em0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 256 to local APIC 0 vector 51 em0: using IRQ 256 for MSI em0: Using an MSI interrupt em0: bpf attached em0: Ethernet address: 00:30:48:93:70:12 pcib3: irq 17 at device 28.5 on pci0 pcib3: allocating non-ISA range 0x5000-0x50ff pcib0: allocated type 4 (0x5000-0x50ff) for rid 1c of pcib3 pcib3: allocating non-ISA range 0x5400-0x54ff pcib0: allocated type 4 (0x5400-0x54ff) for rid 1c of pcib3 pcib3: allocating non-ISA range 0x5800-0x58ff pcib0: allocated type 4 (0x5800-0x58ff) for rid 1c of pcib3 pcib3: allocating non-ISA range 0x5c00-0x5cff pcib0: allocated type 4 (0x5c00-0x5cff) for rid 1c of pcib3 pcib0: allocated type 3 (0xd0200000-0xd02fffff) for rid 20 of pcib3 pcib3: domain 0 pcib3: secondary bus 3 pcib3: subordinate bus 3 pcib3: I/O decode 0x5000-0x5fff pcib3: memory decode 0xd0200000-0xd02fffff pcib3: special decode ISA pci3: on pcib3 pcib3: allocated bus range (3-3) for rid 0 of pci3 pci3: domain=0, physical bus=3 found-> vendor=0x8086, dev=0x109a, revid=0x00 domain=0, bus=3, slot=0, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0107, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=11 powerspec 2 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 32, base 0xd0200000, size 17, enabled pcib3: allocated memory range (0xd0200000-0xd021ffff) for rid 10 of pci0:3:0:0 map[18]: type I/O Port, range 32, base 0x5000, size 5, enabled pcib3: allocated I/O port range (0x5000-0x501f) for rid 18 of pci0:3:0:0 pcib3: matched entry for 3.0.INTA pcib3: slot 0 INTA hardwired to IRQ 17 em1: port 0x5000-0x501f mem 0xd0200000-0xd021ffff irq 17 at device 0.0 on pci3 em1: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 257 to local APIC 0 vector 52 em1: using IRQ 257 for MSI em1: Using an MSI interrupt em1: bpf attached em1: Ethernet address: 00:30:48:93:70:13 pcib4: at device 30.0 on pci0 pcib4: domain 0 pcib4: secondary bus 4 pcib4: subordinate bus 4 pcib4: special decode ISA, subtractive pci4: on pcib4 pcib4: allocated bus range (4-4) for rid 0 of pci4 pci4: domain=0, physical bus=4 isab0: at device 31.0 on pci0 isa0: on isab0 ahci0: port 0x3048-0x304f,0x303c-0x303f,0x3040-0x3047,0x3038-0x303b,0x3020-0x302f mem 0xd0500000-0xd05003ff irq 17 at device 31.2 on pci0 ahci0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 258 to local APIC 0 vector 53 ahci0: using IRQ 258 for MSI ahci0: AHCI v1.10 with 4 3Gbps ports, Port Multiplier not supported ahci0: Caps: 64bit NCQ ALP AL CLO 3Gbps PMD SSC PSC 32cmd 4ports ahcich0: at channel 0 on ahci0 ahcich0: Caps: HPCP ahcich1: at channel 1 on ahci0 ahcich1: Caps: HPCP ahcich2: at channel 2 on ahci0 ahcich2: Caps: HPCP ahcich3: at channel 3 on ahci0 ahcich3: Caps: HPCP pci0: at device 31.3 (no driver attached) acpi_button0: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 atkbd: the current kbd controller command byte 0047 atkbd: keyboard ID 0x41ab (2) kbdc: RESET_KBD return code:00fa kbdc: RESET_KBD status:00aa kbd0 at atkbd0 kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x3d0000 ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 0 vector 54 atkbd0: [GIANT-LOCKED] psm0: unable to allocate IRQ psmcpnp0: irq 12 on acpi0 psm0: current command byte:0047 kbdc: TEST_AUX_PORT status:0000 kbdc: RESET_AUX return code:00fa kbdc: RESET_AUX status:00aa kbdc: RESET_AUX ID:0000 kbdc: RESET_AUX return code:00fa kbdc: RESET_AUX status:00aa kbdc: RESET_AUX ID:0000 psm: status 00 02 64 psm: status 00 00 64 psm: status 00 03 64 psm: status 00 03 64 psm: data 08 00 00 psm: status 01 47 14 psm: status 01 48 a1 psm: status 55 47 55 psm: status 13 02 14 psm: status 3b 47 00 psm: status 10 00 64 psm: status 00 02 64 psm: status 00 02 64 psm0: irq 12 on atkbdc0 ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 0 vector 55 psm0: [GIANT-LOCKED] psm0: model Generic PS/2 mouse, device ID 0-00, 2 buttons psm0: config:00004000, flags:00000008, packet size:3 psm0: syncmask:c0, syncbits:00 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 0 vector 56 uart0: fast interrupt uart0: PPS capture mode: DCDinvalid uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 ioapic0: routing intpin 3 (ISA IRQ 3) to lapic 0 vector 57 uart1: fast interrupt uart1: PPS capture mode: DCDinvalid acpi0: wakeup code va 0xfffffe0096c67000 pa 0x84000 ex_isa_identify() pcib0: allocated type 3 (0xa0000-0xa07ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa0800-0xa0fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa1000-0xa17ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa1800-0xa1fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa2000-0xa27ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa2800-0xa2fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa3000-0xa37ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa3800-0xa3fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa4000-0xa47ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa4800-0xa4fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa5000-0xa57ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa5800-0xa5fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa6000-0xa67ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa6800-0xa6fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa7000-0xa77ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa7800-0xa7fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa8000-0xa87ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa8800-0xa8fff) for rid 0 of orm0 pcib0: allocated type 3 (0xa9000-0xa97ff) for rid 0 of orm0 pcib0: allocated type 3 (0xa9800-0xa9fff) for rid 0 of orm0 pcib0: allocated type 3 (0xaa000-0xaa7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xaa800-0xaafff) for rid 0 of orm0 pcib0: allocated type 3 (0xab000-0xab7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xab800-0xabfff) for rid 0 of orm0 pcib0: allocated type 3 (0xac000-0xac7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xac800-0xacfff) for rid 0 of orm0 pcib0: allocated type 3 (0xad000-0xad7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xad800-0xadfff) for rid 0 of orm0 pcib0: allocated type 3 (0xae000-0xae7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xae800-0xaefff) for rid 0 of orm0 pcib0: allocated type 3 (0xaf000-0xaf7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xaf800-0xaffff) for rid 0 of orm0 pcib0: allocated type 3 (0xb0000-0xb07ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb0800-0xb0fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb1000-0xb17ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb1800-0xb1fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb2000-0xb27ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb2800-0xb2fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb3000-0xb37ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb3800-0xb3fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb4000-0xb47ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb4800-0xb4fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb5000-0xb57ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb5800-0xb5fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb6000-0xb67ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb6800-0xb6fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb7000-0xb77ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb7800-0xb7fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb8000-0xb87ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb8800-0xb8fff) for rid 0 of orm0 pcib0: allocated type 3 (0xb9000-0xb97ff) for rid 0 of orm0 pcib0: allocated type 3 (0xb9800-0xb9fff) for rid 0 of orm0 pcib0: allocated type 3 (0xba000-0xba7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xba800-0xbafff) for rid 0 of orm0 pcib0: allocated type 3 (0xbb000-0xbb7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbb800-0xbbfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbc000-0xbc7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbc800-0xbcfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbd000-0xbd7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbd800-0xbdfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbe000-0xbe7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbe800-0xbefff) for rid 0 of orm0 pcib0: allocated type 3 (0xbf000-0xbf7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbf800-0xbffff) for rid 0 of orm0 pcib0: allocated type 3 (0xd4000-0xd47ff) for rid 2 of orm0 pcib0: allocated type 3 (0xd4800-0xd4fff) for rid 2 of orm0 pcib0: allocated type 3 (0xd5000-0xd57ff) for rid 2 of orm0 pcib0: allocated type 3 (0xd5800-0xd5fff) for rid 2 of orm0 pcib0: allocated type 3 (0xd6000-0xd67ff) for rid 2 of orm0 pcib0: allocated type 3 (0xd6800-0xd6fff) for rid 2 of orm0 pcib0: allocated type 3 (0xd7000-0xd77ff) for rid 2 of orm0 pcib0: allocated type 3 (0xd7800-0xd7fff) for rid 2 of orm0 pcib0: allocated type 3 (0xd8000-0xd87ff) for rid 2 of orm0 pcib0: allocated type 3 (0xd8800-0xd8fff) for rid 2 of orm0 pcib0: allocated type 3 (0xd9000-0xd97ff) for rid 2 of orm0 pcib0: allocated type 3 (0xd9800-0xd9fff) for rid 2 of orm0 pcib0: allocated type 3 (0xda000-0xda7ff) for rid 2 of orm0 pcib0: allocated type 3 (0xda800-0xdafff) for rid 2 of orm0 pcib0: allocated type 3 (0xdb000-0xdb7ff) for rid 2 of orm0 pcib0: allocated type 3 (0xdb800-0xdbfff) for rid 2 of orm0 pcib0: allocated type 3 (0xdc000-0xdc7ff) for rid 2 of orm0 pcib0: allocated type 3 (0xdc800-0xdcfff) for rid 2 of orm0 pcib0: allocated type 3 (0xdd000-0xdd7ff) for rid 2 of orm0 pcib0: allocated type 3 (0xdd800-0xddfff) for rid 2 of orm0 pcib0: allocated type 3 (0xde000-0xde7ff) for rid 2 of orm0 pcib0: allocated type 3 (0xde800-0xdefff) for rid 2 of orm0 pcib0: allocated type 3 (0xdf000-0xdf7ff) for rid 2 of orm0 pcib0: allocated type 3 (0xdf800-0xdffff) for rid 2 of orm0 pcib0: allocated type 3 (0xe0000-0xe07ff) for rid 2 of orm0 pcib0: allocated type 3 (0xe0800-0xe0fff) for rid 2 of orm0 pcib0: allocated type 3 (0xe1000-0xe17ff) for rid 2 of orm0 pcib0: allocated type 3 (0xe1800-0xe1fff) for rid 2 of orm0 pcib0: allocated type 3 (0xe2000-0xe27ff) for rid 2 of orm0 pcib0: allocated type 3 (0xe2800-0xe2fff) for rid 2 of orm0 pcib0: allocated type 3 (0xe3000-0xe37ff) for rid 2 of orm0 pcib0: allocated type 3 (0xe3800-0xe3fff) for rid 2 of orm0 ahc_isa_identify 0: ioport 0xc00 alloc failed ahc_isa_identify 1: ioport 0x1c00 alloc failed ahc_isa_identify 2: ioport 0x2c00 alloc failed ahc_isa_identify 3: ioport 0x3c00 alloc failed ahc_isa_identify 4: ioport 0x4c00 alloc failed ahc_isa_identify 5: ioport 0x5c00 alloc failed ahc_isa_identify 6: ioport 0x6c00 alloc failed ahc_isa_identify 7: ioport 0x7c00 alloc failed ahc_isa_identify 8: ioport 0x8c00 alloc failed ahc_isa_identify 9: ioport 0x9c00 alloc failed ahc_isa_identify 10: ioport 0xac00 alloc failed ahc_isa_identify 11: ioport 0xbc00 alloc failed ahc_isa_identify 12: ioport 0xcc00 alloc failed ahc_isa_identify 13: ioport 0xdc00 alloc failed ahc_isa_identify 14: ioport 0xec00 alloc failed isa_probe_children: disabling PnP devices atkbdc: atkbdc0 already exists; skipping it atrtc: atrtc0 already exists; skipping it attimer: attimer0 already exists; skipping it sc: sc0 already exists; skipping it uart: uart0 already exists; skipping it uart: uart1 already exists; skipping it isa_probe_children: probing non-PnP devices orm0: at iomem 0xc0000-0xcafff,0xcb000-0xcf7ff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sc0: fb0, kbd1, terminal emulator: scteken (teken terminal) vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 pcib0: allocated type 4 (0x3c0-0x3df) for rid 0 of vga0 pcib0: allocated type 3 (0xa0000-0xbffff) for rid 0 of vga0 pcib0: allocated type 4 (0x3f0-0x3f5) for rid 0 of fdc0 pcib0: allocated type 4 (0x3f7-0x3f7) for rid 1 of fdc0 fdc0 failed to probe at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 ppc0: cannot reserve I/O port range ppc0 failed to probe at irq 7 on isa0 wbwd0 failed to probe on isa0 isa_probe_children: probing PnP devices est0: on cpu0 est1: on cpu1 Device configuration finished. procfs registered lapic: Divisor 2, Frequency 99752158 Hz Timecounters tick every 1.000 msec vlan: initialized, using hash tables with chaining pflog0: bpf attached tcp_init: net.inet.tcp.tcbhashsize auto tuned to 16384 lo0: bpf attached hpt27xx: no controller detected. hptrr: no controller detected. hptnr: no controller detected. random: unblocking device. ahcich0: AHCI reset... ahcich0: SATA connect time=100us status=00000123 ahcich0: AHCI reset: device found ahcich1: AHCI reset... ahcich1: SATA connect time=100us status=00000123 ahcich1: AHCI reset: device found ahcich2: AHCI reset... ahcich2: SATA offline status=00000004 ahcich2: AHCI reset: device not found ahcich3: AHCI reset... ahcich3: SATA offline status=00000004 ahcich3: AHCI reset: device not found ahcich0: AHCI reset: device ready after 100ms ahcich1: AHCI reset: device ready after 100ms pass0 at ahcich0 bus 0 scbus0 target 0 lun 0 pass0: ATA-7 SATA 2.x device pass0: Serial Number WD-WCAP94091954 pass0: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) pass0: Command Queueing enabled pass1 at ahcich1 bus 0 scbus1 target 0 lun 0 pass1: ATA-7 SATA 2.x device pass1: Serial Number WD-WCAP94085021 pass1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) pass1: Command Queueing enabled ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: ATA-7 SATA 2.x device ada0: Serial Number WD-WCAP94091954 ada0: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada0: Command Queueing enabled ada0: 152627MB (312581808 512 byte sectors) ada0: Previously was known as ad4 GEOM: new disk ada0 GEOM: new disk ada1 ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 ada1: ATA-7 SATA 2.x device ada1: Serial Number WD-WCAP94085021 ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) ada1: Command Queueing enabled ada1: 152627MB (312581808 512 byte sectors) ada1: Previously was known as ad6 GEOM_RAID: Intel-bbb292a3: Array Intel-bbb292a3 created. GEOM_RAID: Intel-bbb292a3: Disk ada0 state changed from NONE to ACTIVE. GEOM_RAID: Intel-bbb292a3: Subdisk Volume0:0-ada0 state changed from NONE to ACTIVE. GEOM_RAID: Intel-bbb292a3: Disk ada1 state changed from NONE to ACTIVE. GEOM_RAID: Intel-bbb292a3: Subdisk Volume0:1-ada1 state changed from NONE to ACTIVE. GEOM_RAID: Intel-bbb292a3: Array started. GEOM_RAID: Intel-bbb292a3: Volume Volume0 state changed from STARTING to OPTIMAL. GEOM_RAID: Intel-bbb292a3: Provider raid/r0 for volume Volume0 created. SMP: AP CPU #1 Launched! cpu1 AP: ID: 0x01000000 VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 1 vector 48 ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 1 vector 49 ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 1 vector 50 msi: Assigning MSI IRQ 257 to local APIC 1 vector 51 SMP: passed TSC synchronization test TSC timecounter discards lower 1 bit(s) Timecounter "TSC-low" frequency 1097273622 Hz quality 1000 Trying to mount root from ufs:/dev/raid/r0p2 [rw]... start_init: trying /sbin/init lo1: bpf attached em0: Link is up 100 Mbps Full Duplex em0: link state changed to UP pflog0: promiscuous mode enabled --ELM1481653026-17779-0_-- From owner-freebsd-hackers@freebsd.org Tue Dec 13 18:37:00 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 99063C7600D for ; Tue, 13 Dec 2016 18:37:00 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7E9529FA for ; Tue, 13 Dec 2016 18:37:00 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: 258ebc48-c163-11e6-9f67-d3961ed5a660 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id 258ebc48-c163-11e6-9f67-d3961ed5a660; Tue, 13 Dec 2016 18:37:06 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id uBDIapNK001629; Tue, 13 Dec 2016 11:36:51 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1481654211.1889.346.camel@freebsd.org> Subject: Re: system time instability From: Ian Lepore To: Hrant Dadivanyan , Konstantin Belousov Cc: freebsd-hackers@freebsd.org Date: Tue, 13 Dec 2016 11:36:51 -0700 In-Reply-To: References: Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 18:37:00 -0000 On Tue, 2016-12-13 at 22:17 +0400, Hrant Dadivanyan wrote: > > > > On Mon, Dec 12, 2016 at 11:33:59PM +0400, Hrant Dadivanyan wrote: > > > > > > > > > > > On Mon, Dec 12, 2016 at 11:04:08PM +0400, Hrant Dadivanyan > > > > wrote: > > > > > > > > > > Now, when you ask, I start to suspect PPS delivery to uart > > > > > again - cable > > > > > and amplifier, but can't understand how the 100ppm error fits > > > > > into that. > > > > If you disable PPS sync in ntp config, does the machine keep > > > > time adequately ? > > > > > > > Thanks for reminding - yes, I've tried this as well, the issue > > > persists. > > > So uart shouldn't be in charge. > > > > This statement seems to be wrong, look below. > > > > > > > > > > > > > > There might be relatively long pauses when system management > > > > mode handlers > > > > do something in response to hw events.  E.g. if you have USB > > > > emulation of > > > > AT keyboard enabled in BIOS, try to disable that.  And update > > > > the BIOS. > > > The USB is switched off in the BIOS. I've removed all changes in > > > sysctl.conf > > > and nice flag from ntpd, recompiled kernel as following: > > > include         GENERIC > > > options         PPS_SYNC > > > device          pf > > > device          pflog > > > and started over. Dmesg is attached. > > > > > Please show verbose dmesg. > > > I've updated BIOS to the latest one. Verbose dmesg is attached. > > > > > > > > > CPU: Intel(R) Core(TM)2 Duo CPU     E4500  @ 2.20GHz (2194.55-MHz > > > K8-class CPU) > > This is relatively old CPU which is known to have some (minor) > > issues with > > interaction between power saving and cores.  Try the following OS > > config: > > disable deep C states, allow only C1 (there might be some tweaks in > > BIOS, > > if possible, disable the Cn, n > 1, there too); > Have never touched Cx states on servers, it was disabled in BIOS and > sysctl > shows C1 as lowest. Now I've enabled it in BIOS, but didn't touch in > OS: > dev.cpu.1.cx_usage: 100.00% last 31000us > dev.cpu.1.cx_lowest: C1 > dev.cpu.1.cx_supported: C1/1/0 > dev.cpu.0.cx_usage: 100.00% last 5569us > dev.cpu.0.cx_lowest: C1 > dev.cpu.0.cx_supported: C1/1/0 > dev.cpu.0.freq_levels: 2200/35000 2000/31000 1800/27000 1600/23000 > 1400/19000 1200/16000 > dev.cpu.0.freq: 2200 > Is this correct ? > > > > > use LAPIC for event timer (not HPET); > Have disabled HPET in BIOS: > kern.eventtimer.periodic: 0 > kern.eventtimer.timer: LAPIC > kern.eventtimer.idletick: 0 > kern.eventtimer.singlemul: 2 > kern.eventtimer.choice: LAPIC(400) i8254(100) RTC(0) > kern.eventtimer.et.i8254.quality: 100 > kern.eventtimer.et.i8254.frequency: 1193182 > kern.eventtimer.et.i8254.flags: 1 > kern.eventtimer.et.RTC.quality: 0 > kern.eventtimer.et.RTC.frequency: 32768 > kern.eventtimer.et.RTC.flags: 17 > kern.eventtimer.et.LAPIC.quality: 400 > kern.eventtimer.et.LAPIC.frequency: 99751860 > kern.eventtimer.et.LAPIC.flags: 15 > > > > > re-check that you use RDTSC for the timecounter; > kern.timecounter.tsc_shift: 1 > kern.timecounter.smp_tsc_adjust: 0 > kern.timecounter.smp_tsc: 1 > kern.timecounter.invariant_tsc: 1 > kern.timecounter.fast_gettime: 1 > kern.timecounter.tick: 1 > kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) i8254(0) > dummy(-1000000) > kern.timecounter.hardware: TSC-low > kern.timecounter.alloweddeviation: 5 > kern.timecounter.stepwarnings: 0 > kern.timecounter.tc.TSC-low.quality: 1000 > kern.timecounter.tc.TSC-low.frequency: 1097249250 > kern.timecounter.tc.TSC-low.counter: 1335765171 > kern.timecounter.tc.TSC-low.mask: 4294967295 > kern.timecounter.tc.ACPI-fast.quality: 900 > kern.timecounter.tc.ACPI-fast.frequency: 3579545 > kern.timecounter.tc.ACPI-fast.counter: 6343821 > kern.timecounter.tc.ACPI-fast.mask: 16777215 > kern.timecounter.tc.i8254.quality: 0 > kern.timecounter.tc.i8254.frequency: 1193182 > kern.timecounter.tc.i8254.counter: 2701 > kern.timecounter.tc.i8254.mask: 65535 > > > > > do not enable powerd. > > > Never did on servers. > > > > > You might also try the stable/11 kernel, which has more changes WRT > > C-states > > handling and PPS/ntp locking. > > > The server did run for almost a day without PPS and looks stable. I > start > to believe, to my shame, that I did a mistake when testing this > previously. > Then the whole post is wrong and cable seems to be most suspected > part again. > Even now it's hard to understand this wrong behaviour, but anyway ... > > Just replaced the cable with shielded one where each pair has > separate > shield, used dedicated pair for PPS and ground; grounded the shields. > > Thank you Konstantin, thank you Ian ! > Hrant > A bad PPS signal could definitely lead to frequency trouble, if the way the signal is bad involves ringing, or the electrical level floating around the cutoff points for detecting low vs. high level -- you'd get false pulses, and some of them would be close enough to the time of the real pulse that they would make it through the spike/median filters in ntpd.  An early or late pulse looks like a phase step, and several consistant-enough phase steps in the same polling period looks like a frequency step. You mentioned using a 74LS245 bus driver... that can lead to ringing if the load is light, maybe the rs232 port on this new hardware has a much higher input impedance than your old system.  It might be worth adding a series resistor at the computer end to soak up reflections, something in the 30-100 ohm range should work. -- Ian From owner-freebsd-hackers@freebsd.org Tue Dec 13 19:52:10 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 289EAC764C8 for ; Tue, 13 Dec 2016 19:52:10 +0000 (UTC) (envelope-from hrant@dadivanyan.net) Received: from pandora.amnic.net (pandora.amnic.net [IPv6:2001:67c:21c:a610::11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DCF1B1D60; Tue, 13 Dec 2016 19:52:09 +0000 (UTC) (envelope-from hrant@dadivanyan.net) Received: from ran by pandora.amnic.net with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1cGt7P-0006W1-8E; Tue, 13 Dec 2016 23:52:07 +0400 Subject: Re: system time instability In-Reply-To: <1481654211.1889.346.camel@freebsd.org> To: Ian Lepore Date: Tue, 13 Dec 2016 23:52:07 +0400 (AMT) From: Hrant Dadivanyan CC: Konstantin Belousov , freebsd-hackers@freebsd.org X-PGP: https://amnic.net/pgpkeys/hrant.asc X-NCC-RegID: am.isoc X-Mailer: ELM [version 2.4ME+ PL127 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="UTF-8" Message-Id: X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 19:52:10 -0000 [ Charset ISO-8859-1 converted... ] > > The server did run for almost a day without PPS and looks stable. I > > start > > to believe, to my shame, that I did a mistake when testing this > > previously. > > Then the whole post is wrong and cable seems to be most suspected > > part again. > > Even now it's hard to understand this wrong behaviour, but anyway ... > > > > Just replaced the cable with shielded one where each pair has > > separate > > shield, used dedicated pair for PPS and ground; grounded the shields. > > > > Thank you Konstantin, thank you Ian ! > > Hrant > > > > A bad PPS signal could definitely lead to frequency trouble, if the way > the signal is bad involves ringing, or the electrical level floating > around the cutoff points for detecting low vs. high level -- you'd get > false pulses, and some of them would be close enough to the time of the > real pulse that they would make it through the spike/median filters in > ntpd.  An early or late pulse looks like a phase step, and several > consistant-enough phase steps in the same polling period looks like a > frequency step. > > You mentioned using a 74LS245 bus driver... that can lead to ringing if > the load is light, maybe the rs232 port on this new hardware has a much > higher input impedance than your old system.  It might be worth adding > a series resistor at the computer end to soak up reflections, something > in the 30-100 ohm range should work. > Wow, thank you, will try ! > -- Ian -- Hrant Dadivanyan (aka Ran d'Adi) hrant(at)dadivanyan.net /* "Feci quod potui, faciant meliora potentes." */ ran(at)psg.com From owner-freebsd-hackers@freebsd.org Wed Dec 14 07:43:06 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0CB0C76A1B for ; Wed, 14 Dec 2016 07:43:06 +0000 (UTC) (envelope-from meka@tilda.center) Received: from mail.tilda.center (mail.tilda.center [188.226.130.133]) by mx1.freebsd.org (Postfix) with ESMTP id BE6731C07 for ; Wed, 14 Dec 2016 07:43:06 +0000 (UTC) (envelope-from meka@tilda.center) Received: from hal9000.meka.no-ip.org (unknown [87.116.180.115]) by mail.tilda.center (Postfix) with ESMTPSA id 4C661614E8 for ; Wed, 14 Dec 2016 08:42:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=tilda.center; s=mail; t=1481701348; bh=tiltg+JLu/dxahssI3J6n11wQ5r8TslZWNBQkmZI5i8=; h=Date:From:To:Subject:From; b=tcv0UMdHaDhbnAFoZyKTx8CP8cSQyDLOJY2Ylrl4KFtZfr8wOg4MSlvjTmeu7lAhg r6JmYdmrDjDU5x4vxM9WwXXg2Au3ChLhj0U88gsFDnmzXf5B8v6NwUZpYhJ8zDYnL0 AzFfTrW4zmD3kKWzsIQnPU3d+ud2RB0E4MryhaHs= Date: Wed, 14 Dec 2016 08:42:28 +0100 From: Goran =?utf-8?B?TWVracSH?= To: freebsd-hackers@freebsd.org Subject: How to use sem_timedwait? Message-ID: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="w6urz3f5fqxdq7wx" Content-Disposition: inline User-Agent: NeoMutt/20161126 (1.7.1) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 07:43:07 -0000 --w6urz3f5fqxdq7wx Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Hello, I'm trying to understand how sem_timedwait() works. For that I assambled a little program: #include #include #include #include #include int main() { sem_t semaphore; struct timespec ts; clock_gettime(CLOCK_REALTIME, &ts); ts.tv_sec += 3; sem_init(&semaphore, 0, 0); int result = sem_timedwait(&semaphore, &ts); int error = errno; printf("The sem_timedwait() should have ended at %lds and %ldns\n", ts.tv_sec, ts.tv_nsec); clock_gettime(CLOCK_REALTIME, &ts); printf("The sem_timedwait() exited at %lds and %ldns with %d status\n", ts.tv_sec, ts.tv_nsec, result); clock_getres(CLOCK_REALTIME, &ts); printf("Clock resolution is %lds and %ldns\n", ts.tv_sec, ts.tv_nsec); if (result == -1) { switch (error) { case EINVAL: printf("The semaphore does not refer to valid structure\n"); break; case ETIMEDOUT: printf("Timeout occured\n"); break; case EINTR: printf("Interupted\n"); break; default: printf("Unknown error\n"); break; } } } Running it I get the following output: The sem_timedwait() should have ended at 1481700752s and 2050646ns The sem_timedwait() exited at 1481700752s and 3813780ns with -1 status Clock resolution is 0s and 1ns Timeout occured What I want to know is why the number of nanoseconds reported first and second time is not the same if the resolution reported on line 3 is 1ns? It doesn't change much if I use CLOCK_RELATIME_PRECISE. Thank you! Regards, meka --w6urz3f5fqxdq7wx Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE1WIFkXy2ZeMKjjKEWj1TknovrLYFAlhQ9+EACgkQWj1Tknov rLaQIA/+NAqFeNUcpW3k3tevvBxbg8o87fHtgzBfg4MR8cwT36bsaQtIC8KBRoh8 fRepid8Uq72pH+WiPw/NESTpYtAEDlILu4aoYxuUfsyC5YuCMdd01WPNtbP6BIsZ NhC59MQxpq/VKp7u2WrY34I/pX99/7zSremOuyjXfUrvQW2HVs47+ROFFB0Z3YGa rmlMFqZhnDAjUP9+5BfbpPdaHrjAFSMMRmRrf+mIosb0dbdFuf5kvnDSQF9ftip2 u/WrgA9a6TO1N98380YG95YcxbZTg73rM1ctyKVjg8g9RVWsebOflkEe6ZkCqR3B lYuNs3DUf4IRno7tz6xeDP8Dr8qRmeH1fqFtSSsO+I8NJBVwuk9wtKT6IR63dJEZ x0RVklRWI+n6g5zj/mpvUtt9vUPpTjqsaPv3Y1egqKPRNQJdrlTgVzEgx+M7XiNP 5KnGX/cjkhnefKhwUsKV0dimoEKfaMlfdOjY10kOTUddMcZVW8IT0hy6VhKhcj8E vUjltXxI352Y2bA5FMFs5XyLGS20PxWpiMt/oyYo4gXFiNpprL3RqIcuIXjKDNNE q/9LLjB+v93UvDMpzLl3nVoMXV4fO6h6UtIRBPdJB1H1DdqnKusMQegIB+E4KEGG tnjxR8SQp4nznG9qxMmravGUeKYs05couScC321TlEmpU81bnTk= =YaVL -----END PGP SIGNATURE----- --w6urz3f5fqxdq7wx-- From owner-freebsd-hackers@freebsd.org Wed Dec 14 09:24:43 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4E9FEC75FE4 for ; Wed, 14 Dec 2016 09:24:43 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E81B5183A for ; Wed, 14 Dec 2016 09:24:42 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBE9ObWD069237 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Wed, 14 Dec 2016 11:24:38 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBE9ObWD069237 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBE9Obmg069236; Wed, 14 Dec 2016 11:24:37 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 14 Dec 2016 11:24:37 +0200 From: Konstantin Belousov To: Goran Meki?? Cc: freebsd-hackers@freebsd.org Subject: Re: How to use sem_timedwait? Message-ID: <20161214092437.GD94325@kib.kiev.ua> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> User-Agent: Mutt/1.7.2 (2016-11-26) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 09:24:43 -0000 On Wed, Dec 14, 2016 at 08:42:28AM +0100, Goran Meki?? wrote: > Hello, > > I'm trying to understand how sem_timedwait() works. For that I assambled a little program: > > #include > #include > #include > #include > #include > > > int main() { > sem_t semaphore; > struct timespec ts; > clock_gettime(CLOCK_REALTIME, &ts); > ts.tv_sec += 3; > > sem_init(&semaphore, 0, 0); > int result = sem_timedwait(&semaphore, &ts); > int error = errno; > > printf("The sem_timedwait() should have ended at %lds and %ldns\n", ts.tv_sec, ts.tv_nsec); > clock_gettime(CLOCK_REALTIME, &ts); > printf("The sem_timedwait() exited at %lds and %ldns with %d status\n", ts.tv_sec, ts.tv_nsec, result); > > clock_getres(CLOCK_REALTIME, &ts); > printf("Clock resolution is %lds and %ldns\n", ts.tv_sec, ts.tv_nsec); > > if (result == -1) { > switch (error) { > case EINVAL: > printf("The semaphore does not refer to valid structure\n"); > break; > case ETIMEDOUT: > printf("Timeout occured\n"); > break; > case EINTR: > printf("Interupted\n"); > break; > default: > printf("Unknown error\n"); > break; > } > } > } > > > Running it I get the following output: > > The sem_timedwait() should have ended at 1481700752s and 2050646ns > The sem_timedwait() exited at 1481700752s and 3813780ns with -1 status > Clock resolution is 0s and 1ns > Timeout occured > > > What I want to know is why the number of nanoseconds reported first and second time is not the same if the resolution reported on line 3 is 1ns? It doesn't change much if I use CLOCK_RELATIME_PRECISE. Thank you! Because this is how computers work. The declared resolution is the resolution of the clock_gettime(2) call. While you tried to measure the different actions: de-facto you scheduled a wakeup, then put the thread to sleep, then thread was awaken. The system guarantees that the thread is made runnable due to timeout not earlier than the interval specified. After the thread is marked runnable, it should be selected for CPU, then the syscall must complete which put thread to sleep, then userspace must execute enough to call clock_gettime(), then it should return etc. You end up with 1.7ms difference between requested and actual wakeups in this attempt, which is quite good. Note that system also does some micro-optimizations by trying to collapse wakeups close enough to each other, which might also affect occurences of wakeups. From owner-freebsd-hackers@freebsd.org Wed Dec 14 09:51:52 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5CB06C76804 for ; Wed, 14 Dec 2016 09:51:52 +0000 (UTC) (envelope-from meka@tilda.center) Received: from mail.tilda.center (mail.tilda.center [188.226.130.133]) by mx1.freebsd.org (Postfix) with ESMTP id 2702B180B for ; Wed, 14 Dec 2016 09:51:51 +0000 (UTC) (envelope-from meka@tilda.center) Received: from hal9000.meka.no-ip.org (unknown [87.116.180.115]) by mail.tilda.center (Postfix) with ESMTPSA id EF7B75FF89; Wed, 14 Dec 2016 10:51:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=tilda.center; s=mail; t=1481709079; bh=nUl6VpRHsVqy/zJOLJWQTZOmXiD2nPEXxJwovqAqDCA=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=RG2huOhEPrXIFT1vFs/tlvAsiMFvEIcR74pWya0qZi53weudswych4ea70M6ynUOP ZJVr+UqFWiZdenNoCjSDKC1EU+jBkIFt0EmaIAldg/7IT/DDufxBZ9yXAAMjgt55Le Xu3Mdv0yxUz+v5QL2N60Cba2Tdc0a2h6IrBWUDFA= Date: Wed, 14 Dec 2016 10:51:19 +0100 From: Goran =?utf-8?B?TWVracSH?= To: Konstantin Belousov Cc: freebsd-hackers@freebsd.org Subject: Re: How to use sem_timedwait? Message-ID: <20161214095118.4wmt2dtf2cmgangl@hal9000.meka.no-ip.org> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <20161214092437.GD94325@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="54aufmfj3s6lderl" Content-Disposition: inline In-Reply-To: <20161214092437.GD94325@kib.kiev.ua> User-Agent: NeoMutt/20161126 (1.7.1) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 09:51:52 -0000 --54aufmfj3s6lderl Content-Type: text/plain; charset=utf-8 Content-Disposition: inline On Wed, Dec 14, 2016 at 11:24:37AM +0200, Konstantin Belousov wrote: > Because this is how computers work. The declared resolution is the > resolution of the clock_gettime(2) call. While you tried to measure the > different actions: de-facto you scheduled a wakeup, then put the thread > to sleep, then thread was awaken. The system guarantees that the thread > is made runnable due to timeout not earlier than the interval specified. > After the thread is marked runnable, it should be selected for CPU, then > the syscall must complete which put thread to sleep, then userspace must > execute enough to call clock_gettime(), then it should return etc. > > You end up with 1.7ms difference between requested and actual wakeups in > this attempt, which is quite good. > > Note that system also does some micro-optimizations by trying to collapse > wakeups close enough to each other, which might also affect occurences > of wakeups. OK, that sounds logical, but I have a problem now. I need < 1ms precision on this. Is there a way to achieve it? Anyway, thanks for the clarification! Regards, meka --54aufmfj3s6lderl Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE1WIFkXy2ZeMKjjKEWj1TknovrLYFAlhRFhMACgkQWj1Tknov rLY9ZQ/+NIketNBQ/lYgS7oZVh+3SqerDUwHlAZO3R6d47dLGo8WI7vUJtKkzOj5 rUtHCPuGWvA83Ib+eqyuVlAdlsr0BHwjMq0ASzgTwEZMDJwaggOILgkDwifBy0r3 HRMAcUMohlmgmBhsUcjIvA+2rl1ef0BDaQZB3KYMZpRIACFntd+g6FV720FIp15c M+WLcgYcu0Rp7tI1IMEHP3iJbSlWHOpHeFZ+Loj6Rw/sTahB+mqHrHDxsSXgrAT0 hPKnodFiBsuBvh5VxJ2ffNTNynZWeQkyii4rKR0WDO6kCeiFGzS3IwipGEdct13x /w/m1+omEjaIWdE1+4oFoovebTM+7g21oaNIvOTX2UX/cLTn24uPb4e8zwVeCKYb n6h2jCICnb6sOvxzUtcecLOYZxvYGcZwgYYyIXRJuBLpj3X4x1ulQcb3+5uj0+nm YZ9I29y1icgZvCY2YFX9FRIlhoSWUdrOsqYAx1FQLFwE2JdFbLLZY1FzR3oMx8WX L7TeIgrdDgwwqgTUVW+nYCVKjiWRHKOnyKrK14ALngdeB9onOSYn6mBu9mFvj3N1 QRYcBa+03TPyrYz+tZQI4CRQysx/5Y3WXb0P7sLxkaGZ3PFYUT32krZ9/xBm3gWu bl+Us3uABHVPlgoYntHagS9vU/np2ppTEjbNWxGKr1sk85PL3O0= =xzAu -----END PGP SIGNATURE----- --54aufmfj3s6lderl-- From owner-freebsd-hackers@freebsd.org Wed Dec 14 13:21:25 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D3148C770EA for ; Wed, 14 Dec 2016 13:21:25 +0000 (UTC) (envelope-from Alexander@leidinger.net) Received: from mailgate.Leidinger.net (mailgate.leidinger.net [IPv6:2a00:1828:2000:375::1:5]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6EE46144A for ; Wed, 14 Dec 2016 13:21:25 +0000 (UTC) (envelope-from Alexander@leidinger.net) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=leidinger.net; s=outgoing-alex; t=1481721649; bh=dmDe1pObz4otNTsUVRRXGhOsYPap+88xqHB08btwScc=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=3Oax7g5rNO92DZke7zFm9a8MjWTJyOH7N9x/RYDyTT0h03k3vT9NfSWdwhIVF5j6+ YLHVKwLI2VDMqOilsdy9B2zdjbfU4wfuHynovXDoyw2SzCDjLc3GMzPa+qgTOmyVxc N/lESPWL8xgbqbm3adO4XTwzIvpsvlipLxkYpcla6oLqzxagSYTuXHrBjN7kGhejQx anxHfKQJhCkvyxOQ2DcY9TOeQ06O8EpVJLdQbI628GsUQpNoQvUPE7rapmyzDKiFKA KoFmVPA4YTWKmWV+inDvAgjhNVBT0UCHphDKA30xBcyYfaAHDL7ZA6s5K+EUdCLjRw WRyPg/wKtG5xQ== DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=leidinger.net; s=outgoing-alex; t=1481721682; bh=dmDe1pObz4otNTsUVRRXGhOsYPap+88xqHB08btwScc=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=Zej3c0Ohh5YJiTmeLO76NCoQhf4Qgw4Kn1mQQLPfeKBaW9JwbH0lEWJfFg0ceqrJN 5Sb+XNGZAdRPFlF2WofBRmvTrjYjBPHp0fKTXPqeN1UgUpco/N4zRimEyiFj7QMNoU pUouQJGhcW2/puVb5S/U4g2Yyz2gXYK3XX0Lf3IfLDPDO7JYY3o8GafMtLUJnz8f98 dgeLvlRAAmFCKCMRH2ISsedqBtRx7wWaJ+qiSLMtm3KVP6+3MXAvrANcc0udjBZjxI dXT9ySO58xIh4KnR8Yyjo3usHQmyTIu9Y2kjWBeQgJwlwfdd4vi9qqijlIRIQtrMwz U3Rrx55dgBqlg== Date: Wed, 14 Dec 2016 14:20:48 +0100 Message-ID: <20161214142048.Horde.JOZn2cO5FyhzaoCSGj30FT1@webmail.leidinger.net> From: Alexander Leidinger To: Adrian Chadd Cc: Warner Losh , FreeBSD Hackers , Ed Schouten Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> <20161213052759.GP61036@server.rulingia.com> In-Reply-To: User-Agent: Horde Application Framework 5 Content-Type: multipart/signed; boundary="=_HNDVj2HQMLj3zBlwWfVC7Xq"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 X-Mailman-Approved-At: Wed, 14 Dec 2016 13:39:23 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 13:21:25 -0000 This message is in MIME format and has been PGP signed. --=_HNDVj2HQMLj3zBlwWfVC7Xq Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Quoting Adrian Chadd (from Mon, 12 Dec 2016=20=20 23:32:00=20-0800): > On 12 December 2016 at 22:03, Warner Losh wrote: >> On Mon, Dec 12, 2016 at 10:49 PM, Adrian Chadd=20=20 >>=20 wrote: >>> If only we had a mechanism for these kinds of sensors to register, so >>> the API they'd be using would be one that lead itself to edge/level >>> triggered comparisons, versus polling. >>> >>> Wait, didn't we /have/ one of those up for review a few years ago, and >>> it wasn't merged in? >> >> Didn't we have people willing to work with the folks that put this up >> when to address the defects in the review? Oh, wait, they never wanted >> to work to fix the problems, even with many offers of help to do so. >> >> My offer still stands, btw. > > Heh, this wasn't a troll? :) I should've been clearer! > > Does anyone remember where the ye olde framework was? Original: - Google Summer of Code 2007. - I don't find the repo in svnweb right now=20=20 (https://svnweb.freebsd.org/socsvn/).=20Where do we have the things from=20= =20 before=202011? There was at least in the past someone keeping it at least up-to-date,=20= =20 but=20I don't remember right now who/where. If someone is interested in the mail-thread of the past... available=20=20 on=20request. Bye, Alexander. --=20 http://www.Leidinger.net=20Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF --=_HNDVj2HQMLj3zBlwWfVC7Xq Content-Type: application/pgp-signature Content-Description: Digitale PGP-Signatur Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iQIcBAABAgAGBQJYUUcwAAoJEKrxQhqFIICEHwoP/3bazNZyEGTOoJIp43lXrZuU MPHWlzJm/3tI7QyWLXHTjD7ANDSTcie8Q6D7mivR6oTotgZh1xvGonej3m19oxaz vR+M0ygRPnNMxyZMG6AGNLfATxEG/0TuUM8EtMc/N2ViojVmq3Zf3h4njrmtao6z xAlexNT93LykmPe31nlqmCnPCcHpgrsXtBOJa5TcM1tLlNnusz2YuW8eM+x54XEp z/fBkHG42D11UgC/gLINRxWoeeadzS64lwbQRIQWnWwt++nSbIy9ak2JPZz5bnfs TOA7wemAZHI474rdAghuXg5KSRqqytewSTiOHM8/DuRKEoxB7WTXJRWKCUAjaIFs NhbLsHcWnMT0a6cIWOAaUTzqt8c83CDvh18qfk69gIWCDsxCgVFnpWidbVSpTPw3 qWppyJjwYQEERgYFWFCdE9uHfHPuNGTKi2GSBS6HX8amq7EeuypwfelUaBcMVVX+ p0tkR+/Iz8CsnH+l/X29g9LeVfPt2sj6TQcmAoHHr1lpvJFpv7gxp7H5iy9mZf79 r2eerlpKXy1/DjXTNXN8fp4ov4eFlL9t7s+YFL5nKYQjbTnDDfohV3D7x7G6Ltz2 GFpQ73Ag2xcQjERJZG+APSdU+qbt7gHfVtFHGO3s/bjJdnUK5OxzTph1Y5cWpUUK NLzVMMDFhz3ARJI3NvAn =DHtZ -----END PGP SIGNATURE----- --=_HNDVj2HQMLj3zBlwWfVC7Xq-- From owner-freebsd-hackers@freebsd.org Wed Dec 14 15:11:44 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8D573C8040E for ; Wed, 14 Dec 2016 15:11:44 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: from mail-yb0-x22e.google.com (mail-yb0-x22e.google.com [IPv6:2607:f8b0:4002:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44AEF10AD for ; Wed, 14 Dec 2016 15:11:44 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: by mail-yb0-x22e.google.com with SMTP id d128so17756615ybh.2 for ; Wed, 14 Dec 2016 07:11:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuxi-nl.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=wTYap+ifONkKxWB3ZkjZDrkpwIQWWQwvXdiUesk7ILc=; b=HAno+UMCAUCN0tj5M48iTMrMEcodRRt2506uGQp+eRkgccd/bcNty9/feYvwOomAHU C4Nuebui3herOrjmLBSjbTXTC3uLC5enBwUKkS9vIDb3cVDlsShnfsFW5gSUOcVgL0uE NELfk9RaWWlfLvg+Sex4E6q9uT7DWU++oXkFNqexyXD8HKmdEE9hBra0tcKAP401vk14 nUVmJXOOI0SWjWfkdzWqDm9cyi6YQ9FnQuOzhOPB6UN+qb041xoMYN1tb3ZEeVU3zaXA s99k/Dc6dRD6gK2P+J7g4xjm2edtMG4QK987947x71D6aAYli0b4EwYsn7LGKRppDl/L Q5FQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=wTYap+ifONkKxWB3ZkjZDrkpwIQWWQwvXdiUesk7ILc=; b=BYLsqYrim0db8h3RIJ7Vslocg95kzfeXnm9OKJnYcq1zVgaYRyPi2L1gcQOFBsEhUS AhN1pXJhpJlqet02G5Ba7J8NZ8W1n3ZcP9QbYznqCnXUJszXHqe5yojoNihvG3RkiRYU RQk6/X5w2gBbdfRakFdDQhgfG8vNEL2pvwPN0f5hIn9h1wSa57SOHj++lO+0A+160kEB 1jI1Qqq2gYbQbnUNV9+RszNNWMGwlkRVQctXtcSWyVt6mrNXdP5vkJ+8WU6gHowCvhgX pfAVD2X0EHF6WtWCN05huya0/lXPp7bNx0EP/z+GxtfoDx7u8n4tOVkADGtd0ZUl9eSw LnjQ== X-Gm-Message-State: AKaTC02JPfks3bi6FRUmy8ssJ+07+kSncixBQcKoCGiSfpDlvQoLrN2asLpxldPuWF+g/2DzXRUgJSC2190MtQ== X-Received: by 10.13.221.215 with SMTP id g206mr108365167ywe.350.1481728303554; Wed, 14 Dec 2016 07:11:43 -0800 (PST) MIME-Version: 1.0 Received: by 10.129.0.212 with HTTP; Wed, 14 Dec 2016 07:11:13 -0800 (PST) In-Reply-To: References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> From: Ed Schouten Date: Wed, 14 Dec 2016 16:11:13 +0100 Message-ID: Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: Warner Losh Cc: Jonathan Anderson , FreeBSD Hackers , Allan Jude Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 15:11:44 -0000 2016-12-13 16:08 GMT+01:00 Ed Schouten : > Anyway, I just finished implementing support for attaching labels to > sysctl nodes: > > https://reviews.freebsd.org/D8775 After cem@ and rpokala@ were kind enough to review my changes, I've committed them to HEAD. Thanks! In the meantime, I've published a second code review for importing the actual prometheus_sysctl_exporter: https://reviews.freebsd.org/D8792 -- Ed Schouten Nuxi, 's-Hertogenbosch, the Netherlands KvK-nr.: 62051717 From owner-freebsd-hackers@freebsd.org Wed Dec 14 15:54:55 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 022FBC662C6 for ; Wed, 14 Dec 2016 15:54:55 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id E1282F03 for ; Wed, 14 Dec 2016 15:54:54 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: by mailman.ysv.freebsd.org (Postfix) id DD7EEC662C5; Wed, 14 Dec 2016 15:54:54 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DD089C662C4 for ; Wed, 14 Dec 2016 15:54:54 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from weser.webweaving.org (weser.webweaving.org [148.251.234.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.webweaving.org", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 61E6AF02 for ; Wed, 14 Dec 2016 15:54:53 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from beeb.leiden.webweaving.org (5ED29A06.cm-7-3c.dynamic.ziggo.nl [94.210.154.6]) (authenticated bits=0) by weser.webweaving.org (8.15.2/8.15.2) with ESMTPSA id uBEFR03t090788 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Wed, 14 Dec 2016 16:27:00 +0100 (CET) (envelope-from dirkx@webweaving.org) X-Authentication-Warning: weser.webweaving.org: Host 5ED29A06.cm-7-3c.dynamic.ziggo.nl [94.210.154.6] claimed to be beeb.leiden.webweaving.org From: Dirk-Willem van Gulik Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: ZFS - directory entry Message-Id: Date: Wed, 14 Dec 2016 16:27:00 +0100 To: FreeBSD Hackers X-Mailer: Apple Mail (2.3259) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (weser.webweaving.org [148.251.234.232]); Wed, 14 Dec 2016 16:27:00 +0100 (CET) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 15:54:55 -0000 A rather odd directory entry (in /root, the home dir of root/toor) = appeared on a bog standard FreeBSD 10.2 (p18) lightly loaded machine = under ZFS during/post a backup: $ ls -la /root | tail -q ---------- 1 root wheel 9223372036854775807 Jan 1 1970 = ?%+?kD?H???x,?5?Dh;*s!?h???jw??????\h?:????????``?13?@?????OA????????Puux?= ??? Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B07F0C66DA1 for ; Wed, 14 Dec 2016 16:14:32 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 8D4591730 for ; Wed, 14 Dec 2016 16:14:32 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 899D7C66D9D; Wed, 14 Dec 2016 16:14:32 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8945EC66D9A for ; Wed, 14 Dec 2016 16:14:32 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qk0-x244.google.com (mail-qk0-x244.google.com [IPv6:2607:f8b0:400d:c09::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4704E172A for ; Wed, 14 Dec 2016 16:14:32 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-qk0-x244.google.com with SMTP id h201so3732301qke.3 for ; Wed, 14 Dec 2016 08:14:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=Tk8r9S72fn6zb0eFlJFjSohT3yqzHe+mnoDOKiQvLWk=; b=mXtoCMHWHO2WqkPc8DN/xt/dZR1KiVbzkDwaL/rL4hOjFb5XrKMDaI8mXSprhbS9pv /JSGrgIR/G3qgprpEejU5FqRTtWq6eUmf+0s8IQxGz+fpw9BQkPPbR6uUYORmuPM4Zw1 BdQcg5CsjaUGsjQfvco+fkndNvHfjkguhlRiOfbSs38WaVbFXV7V2zdSe/nsKuAnZhtj xKQEmwoztHUHZSWr6tq8cNHM7W6Ip3QHPiWuQvBqk1GjLT0QBQ4KxYjVw0Ez25A9cro9 /DQELC/CfSxxzH+2FrOrn1/c4XQpNV3wsWrR3cUZZYubNu0THsoy8y3VfD4aJxB54pmL E+gA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=Tk8r9S72fn6zb0eFlJFjSohT3yqzHe+mnoDOKiQvLWk=; b=LkbVrkxF+h15gMecXmOvIlzsq+HisRXqY/RJGnIs93hx0Wzvzr3QjTOQoaktz8Kzc6 KHDgvYG2Z0YD+cQHb2hsgSaOQcxtCZr+LSN64BqO02ePUtR38Cjs8yz8bv2CtjI5bG3d ccyOWpPV7s7jdUuM8E9XiVi0+vnFRRUHGWDx36ASIJOJ2cfC6orBMFv9I2Ea3U4IzV7W ez5Zdjzdln5IaSlAT81w6qie+9FCXFx6amDpnlzVrqfqGmUtxieZ3H0RwBJ1COPsfxtU 0kMVSrKC4oPOs6FE7WkdIlDnwt4bux5KVw49BescRi8tlXk8N+U3InJqQ8E1ovvQdoG/ UpUA== X-Gm-Message-State: AKaTC00bfUwL8mJ52M5BOPLHd3gH9wne06SvMSA+Z3s9wY02JoJXCMgg9XGB7pTiZl/WVaSh4tH2ZXvWKpSApQ== X-Received: by 10.55.140.132 with SMTP id o126mr100165561qkd.227.1481732071299; Wed, 14 Dec 2016 08:14:31 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.12.174.145 with HTTP; Wed, 14 Dec 2016 08:14:30 -0800 (PST) In-Reply-To: References: From: Alan Somers Date: Wed, 14 Dec 2016 09:14:30 -0700 X-Google-Sender-Auth: 7k8N1I3Gnu4m-OCyTVidUTWNfKU Message-ID: Subject: Re: ZFS - directory entry To: Dirk-Willem van Gulik Cc: FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 16:14:32 -0000 On Wed, Dec 14, 2016 at 8:27 AM, Dirk-Willem van Gulik wrote: > A rather odd directory entry (in /root, the home dir of root/toor) appear= ed on a bog standard FreeBSD 10.2 (p18) lightly loaded machine under ZFS du= ring/post a backup: > > $ ls -la /root | tail -q > ---------- 1 root wheel 9223372036854775807 Jan 1 1970 ?%+?kD?H???x= ,?5?Dh;*s!?h???jw??????\h?:????????``?13?@?????OA????????Puux???? > OS and ZFS is installed with a bog standard sysinstall. =E2=80=98SMART=E2= =80=99 nor smartd have reported anything. nothing in dmesg, syslog of boot = log. Any suggestions as how to debug or get to the root of this ? > > And in particular - what is a risk of a reboot (to get a kernel with debu= g, etc) causing the issue to =E2=80=98go away=E2=80=99 - and hence stopping= the forensic ? > > Dw. > > sudo zpool list -v > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH A= LTROOT > tank 25.2T 9.27T 16.0T - 17% 36% 1.53x ONLINE - > raidz3 25.2T 9.27T 16.0T - 17% 36% > ada0p3 - - - - - - > ada1p3 - - - - - - > ada2p3 - - - - - - > ada3p3 - - - - - - > ada4p3 - - - - - - > ada5p3 - - - - - - > ada6p3 - - - - - - Two things to try: 1) zpool scrub. This will reveal any corrupt metadata objects 2) Maybe the filename is created in an encoding not supported by your current terminal. Try "LANG=3Den_US.UTF-8 ls -l" 3) Use zdb to examine the file. First, do "ls -li /root" to get the object id. It's the same as the inode number. Then, assuming /root is in the tank/root filesystem, do "zdb -ddddd tank/root ". That might reveal some clues. -Alan From owner-freebsd-hackers@freebsd.org Wed Dec 14 16:38:16 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 90876C80896 for ; Wed, 14 Dec 2016 16:38:16 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [81.2.117.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 371041F7B for ; Wed, 14 Dec 2016 16:38:16 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from host-4-75.office.adestra.com (unknown [85.199.232.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: m.seaman@infracaninophile.co.uk) by smtp.infracaninophile.co.uk (Postfix) with ESMTPSA id E05E27BBE for ; Wed, 14 Dec 2016 16:38:08 +0000 (UTC) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none header.from=FreeBSD.org Authentication-Results: smtp.infracaninophile.co.uk/E05E27BBE; dkim=none; dkim-atps=neutral Subject: Re: ZFS - directory entry To: freebsd-hackers@freebsd.org References: From: Matthew Seaman Message-ID: <08cf61a1-d1ff-66e9-81e7-371bb47eec34@FreeBSD.org> Date: Wed, 14 Dec 2016 16:38:01 +0000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="COjCLCtdVp4jrOpsl240HElpCKFDuVJmC" X-Spam-Status: No, score=0.9 required=5.0 tests=BAYES_00,RCVD_IN_RP_RNBL, RDNS_NONE,SPF_SOFTFAIL autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on smtp.infracaninophile.co.uk X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 16:38:16 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --COjCLCtdVp4jrOpsl240HElpCKFDuVJmC Content-Type: multipart/mixed; boundary="WVX3CJwwNL5Lgqw5HAEm83cljO1aX9LhG"; protected-headers="v1" From: Matthew Seaman To: freebsd-hackers@freebsd.org Message-ID: <08cf61a1-d1ff-66e9-81e7-371bb47eec34@FreeBSD.org> Subject: Re: ZFS - directory entry References: In-Reply-To: --WVX3CJwwNL5Lgqw5HAEm83cljO1aX9LhG Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 2016/12/14 16:14, Alan Somers wrote: > On Wed, Dec 14, 2016 at 8:27 AM, Dirk-Willem van Gulik > wrote: >> A rather odd directory entry (in /root, the home dir of root/toor) app= eared on a bog standard FreeBSD 10.2 (p18) lightly loaded machine under Z= FS during/post a backup: >> >> $ ls -la /root | tail -q >> ---------- 1 root wheel 9223372036854775807 Jan 1 1970 ?%+?kD?H?= ??x,?5?Dh;*s!?h???jw??????\h?:????????``?13?@?????OA????????Puux????> >> OS and ZFS is installed with a bog standard sysinstall. =E2=80=98SMART= =E2=80=99 nor smartd have reported anything. nothing in dmesg, syslog of = boot log. Any suggestions as how to debug or get to the root of this ? >> >> And in particular - what is a risk of a reboot (to get a kernel with d= ebug, etc) causing the issue to =E2=80=98go away=E2=80=99 - and hence sto= pping the forensic ? >> >> Dw. >> >> sudo zpool list -v >> NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH= ALTROOT >> tank 25.2T 9.27T 16.0T - 17% 36% 1.53x ONLINE= - >> raidz3 25.2T 9.27T 16.0T - 17% 36% >> ada0p3 - - - - - - >> ada1p3 - - - - - - >> ada2p3 - - - - - - >> ada3p3 - - - - - - >> ada4p3 - - - - - - >> ada5p3 - - - - - - >> ada6p3 - - - - - - >=20 > Two things to try: > 1) zpool scrub. This will reveal any corrupt metadata objects > 2) Maybe the filename is created in an encoding not supported by your > current terminal. Try "LANG=3Den_US.UTF-8 ls -l" > 3) Use zdb to examine the file. First, do "ls -li /root" to get the > object id. It's the same as the inode number. Then, assuming /root > is in the tank/root filesystem, do "zdb -ddddd tank/root ". > That might reveal some clues. >=20 That file size is one less than 8192 PiB or 0x7fffffffffffffff bytes. That would be pretty impressive, if it was real. Cheers, Matthew --WVX3CJwwNL5Lgqw5HAEm83cljO1aX9LhG-- --COjCLCtdVp4jrOpsl240HElpCKFDuVJmC Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQJ8BAEBCgBmBQJYUXVvXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQxOUYxNTRFQ0JGMTEyRTUwNTQ0RTNGMzAw MDUxM0YxMEUwQTlFNEU3AAoJEABRPxDgqeTnhBAP/iCNU2aU5UK4mtALfoFwH8GB Jsd7XyISWPMvNABG8jYQH1BWq9f90QpnuVsGmBm8BIFpxR1jIQaYBF6QHxdaRGLv nfOXbuIdpvTbbH7JMYkbZQzelaPdQIw6aOtKgoJ/4j7Hs97Bii4HpIzLhJKjp5HB uotp7sXzWsS6MhmDW922X4b1I0HekfuMQe1K7xXeLSPkRYzYybR8HpaY11IqSQp8 eaZYZOtrO62XI4rzcNrUG4utDyVP50/VVAwauMTLY4evweVn8CwhMGQz08qXvQ4a YjFqaLIsPYq49FBe4p23ih+zSiIUC1yfG8ihte9ZVVk5JdDAS4MVZhTEKU948eRT H5Y9IDdovNdTVpif3dCBICH4NSP6R4U7Zt8yvyHe1eLblkbJDvsL4wYWUkU2YIbG n5hlcwQwX7560bjaNczlvIY0MT46HQZj4Vr6nVMr6XIy7cBFRyQgCQHDMdMlluga 1kBmI32NBejcBEGmUN34lMC+1KGp0C++1DMeMSKP7D5KU3mwDY7nTdPxJLCx1H/f YrBfotI/w7u6MWTFotuWinNjt+IZt4ZPDSQreH4QCmAoXrXptz5SFiWFZH1yO8OB vPrhGzIDaTS0KPHg8P9X0ICBkHWT+/yehbUQGjIGv2vmCRpcrVG3NSkQPO0N+sJf B0nBZO3faXeXMhD2Ibp0 =FdXz -----END PGP SIGNATURE----- --COjCLCtdVp4jrOpsl240HElpCKFDuVJmC-- From owner-freebsd-hackers@freebsd.org Wed Dec 14 17:17:17 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6676AC778E2 for ; Wed, 14 Dec 2016 17:17:17 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4F12D6CA for ; Wed, 14 Dec 2016 17:17:17 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: by mailman.ysv.freebsd.org (Postfix) id 4E58FC778E1; Wed, 14 Dec 2016 17:17:17 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4DFACC778E0 for ; Wed, 14 Dec 2016 17:17:17 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from weser.webweaving.org (weser.webweaving.org [148.251.234.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.webweaving.org", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EE6936C9 for ; Wed, 14 Dec 2016 17:17:16 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from beeb.leiden.webweaving.org (5ED29A06.cm-7-3c.dynamic.ziggo.nl [94.210.154.6]) (authenticated bits=0) by weser.webweaving.org (8.15.2/8.15.2) with ESMTPSA id uBEHFeJf095674 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 14 Dec 2016 18:15:55 +0100 (CET) (envelope-from dirkx@webweaving.org) X-Authentication-Warning: weser.webweaving.org: Host 5ED29A06.cm-7-3c.dynamic.ziggo.nl [94.210.154.6] claimed to be beeb.leiden.webweaving.org Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Re: ZFS - directory entry From: Dirk-Willem van Gulik In-Reply-To: Date: Wed, 14 Dec 2016 18:15:40 +0100 Cc: FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Alan Somers X-Mailer: Apple Mail (2.3259) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (weser.webweaving.org [148.251.234.232]); Wed, 14 Dec 2016 18:15:56 +0100 (CET) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 17:17:17 -0000 > On 14 Dec 2016, at 17:14, Alan Somers wrote: >=20 > On Wed, Dec 14, 2016 at 8:27 AM, Dirk-Willem van Gulik > wrote: >> A rather odd directory entry (in /root, the home dir of root/toor) = appeared on a bog standard FreeBSD 10.2 (p18) lightly loaded machine = under ZFS during/post a backup: >>=20 >> $ ls -la /root | tail -q >> ---------- 1 root wheel 9223372036854775807 Jan 1 1970 = ?%+?kD?H???x,?5?Dh;*s!?h???jw??????\h?:????????``?13?@?????OA????????Puux?= ???>=20 >> OS and ZFS is installed with a bog standard sysinstall. =E2=80=98SMART=E2= =80=99 nor smartd have reported anything. nothing in dmesg, syslog of = boot log. Any suggestions as how to debug or get to the root of this ? >>=20 >> And in particular - what is a risk of a reboot (to get a kernel with = debug, etc) causing the issue to =E2=80=98go away=E2=80=99 - and hence = stopping the forensic ? >>=20 >> Dw. >>=20 >> sudo zpool list -v >> NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP = HEALTH ALTROOT >> tank 25.2T 9.27T 16.0T - 17% 36% 1.53x = ONLINE - >> raidz3 25.2T 9.27T 16.0T - 17% 36% >> ada0p3 - - - - - - >> ada1p3 - - - - - - >> ada2p3 - - - - - - >> ada3p3 - - - - - - >> ada4p3 - - - - - - >> ada5p3 - - - - - - >> ada6p3 - - - - - - Most wonderful - I did not know about the inode/zdb magic. Thanks! > Two things to try: > 1) zpool scrub. This will reveal any corrupt metadata objects Ok - some 300 hours to go :) So am now trying figure out why it is = running at just 8M/s (prefetch_disable=3D1, vfs.zfs.scrub_delay=3D0). > 2) Maybe the filename is created in an encoding not supported by your > current terminal. Try "LANG=3Den_US.UTF-8 ls -l=E2=80=9D No cookie (and not overly likely - barebone install which is not visible = to anything =E2=80=98modern=E2=80=99 but ssh et.al.). > 3) Use zdb to examine the file. First, do "ls -li /root" to get the > object id. It's the same as the inode number. Then, assuming /root > is in the tank/root filesystem, do "zdb -ddddd tank/root ". > That might reveal some clues. A: zdb -ddddd tank/root 7426414 gives; after a 50 second pause (pre/during =E2=80=98zpool scrub=E2=80=99)= : Dataset tank/root [ZPL], ID 40, cr_txg 6, 902M, 14669 objects, = rootbp DVA[0]=3D<0:4c000:4000> DVA[1]=3D<0:4c00004c000:4000> [L0 DMU = objset] fletcher4 uncompressed LE contiguous unique double = size=3D800L/800P birth=3D225L/225P fill=3D14669 = cksum=3D9c7252c3b:ad096bfa68f:7b6298f1d2648:4235b444c02eba0 Object lvl iblk dblk dsize lsize %full type zdb: dmu_bonus_hold(7426414) failed, errno 2 So I guess I should wait for the scrub to complete. Cannot recall scrub = to be this slow. Dw. From owner-freebsd-hackers@freebsd.org Wed Dec 14 18:52:20 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B3F14C801A0 for ; Wed, 14 Dec 2016 18:52:20 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 99F43169D for ; Wed, 14 Dec 2016 18:52:20 +0000 (UTC) (envelope-from peter@rulingia.com) Received: by mailman.ysv.freebsd.org (Postfix) id 993F6C8019F; Wed, 14 Dec 2016 18:52:20 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 98DE0C8019E for ; Wed, 14 Dec 2016 18:52:20 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (vps.rulingia.com [103.243.244.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.rulingia.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2C0C9169B for ; Wed, 14 Dec 2016 18:52:19 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (ppp59-167-167-3.static.internode.on.net [59.167.167.3]) by vps.rulingia.com (8.15.2/8.15.2) with ESMTPS id uBEIq5u1092014 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 15 Dec 2016 05:52:14 +1100 (AEDT) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.15.2/8.15.2) with ESMTPS id uBEIpxbr031967 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 15 Dec 2016 05:51:59 +1100 (AEDT) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.15.2/8.15.2/Submit) id uBEIptiU031966; Thu, 15 Dec 2016 05:51:55 +1100 (AEDT) (envelope-from peter) Date: Thu, 15 Dec 2016 05:51:54 +1100 From: Peter Jeremy To: Dirk-Willem van Gulik Cc: FreeBSD Hackers Subject: Re: ZFS - directory entry Message-ID: <20161214185154.GT61036@server.rulingia.com> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="brEuL7wsLY8+TuWz" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.7.1 (2016-10-04) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 18:52:20 -0000 --brEuL7wsLY8+TuWz Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2016-Dec-14 16:27:00 +0100, Dirk-Willem van Gulik = wrote: >A rather odd directory entry (in /root, the home dir of root/toor) appeare= d on a bog standard FreeBSD 10.2 (p18) lightly loaded machine under ZFS dur= ing/post a backup: > >$ ls -la /root | tail -q >---------- 1 root wheel 9223372036854775807 Jan 1 1970 ?%+?kD?H???x,= ?5?Dh;*s!?h???jw??????\h?:????????``?13?@?????OA????????Puux???? >OS and ZFS is installed with a bog standard sysinstall. =E2=80=98SMART=E2= =80=99 nor smartd have reported anything. nothing in dmesg, syslog of boot = log. Any suggestions as how to debug or get to the root of this ?=20 > >And in particular - what is a risk of a reboot (to get a kernel with debug= , etc) causing the issue to =E2=80=98go away=E2=80=99 - and hence stopping = the forensic ? Do you have ECC RAM? If not, it's possible this is an artifact of some RAM corruption, rather than on-disk corruption. I'm surprised by the slow scrub, though they are very slow disks. You might like to use gstat or zpool iostat to see if one of the disks is slower than the others - indicating a possible problem with it. --=20 Peter Jeremy --brEuL7wsLY8+TuWz Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJYUZTKXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFRUIyOTg2QzMwNjcxRTc0RTY1QzIyN0Ux NkE1OTdBMEU0QTIwQjM0AAoJEBall6Dkogs0uCAP/jUkIaHMEwy3k+CobYxtos/s dkPG6cuxfgts47LJDDOd4K9P+8m946JDT6b1Pfia/KncSPZL/x86ie7gtTpCl6x5 TUC47FNXQlQKSjlw59p+3/lbYY/XTAfEWbL3QOR7A0vPc3Hie/uHsoco2WUxfAMF V56FVYYIqV8Qh68iw3VW0WDxqwMrkdXFc6+Lqdwqx5jMX9NEd5z965+0yH87blW/ 3nnghOfMSiiOiKJkC3M0vd2oposoAxXoPWuRc05Acd6fWV5Q9bLkl3HSMbRc4rnO Mt+ycBtCB+Kv4Pdw5G0N91DtS3V/5m0MTGS/7Bl7/0OMvz0XzKU95EvauZzovgXR nRzbMWRfcG08ejT+uHLxKXraXBLR8HYJb7NW6KoKA8ZF2CJszzb3qtNgFkyjvroR VLJq6pp6VXk3eutC9arPIdFH/16324FFCqY0xZ9+a3ErjMv2vTG4NWHzcZOHOM2S pAq0Ezg5RMqUWJ9EcEEsBFgE/GcCc+w0GohWrl85ufQIaZcgBhm7Dvwxj7kEvGwx /2uFpnnj2ub41gcxvYJPwGib86Hkofrqb7859MdHXfeEUjapHv725yIEN6HZIAqE tt52UpOGHXwHWWComvM6nZPpCnU8VG/DADBGvlqInD747ecdiLeVxhROkc9aitj2 wP3wiEDPb8rEnnLQ+SjM =Yz5B -----END PGP SIGNATURE----- --brEuL7wsLY8+TuWz-- From owner-freebsd-hackers@freebsd.org Wed Dec 14 19:02:32 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 26938C804B0 for ; Wed, 14 Dec 2016 19:02:32 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0FBB21ECB for ; Wed, 14 Dec 2016 19:02:32 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: by mailman.ysv.freebsd.org (Postfix) id 0F1FBC804AF; Wed, 14 Dec 2016 19:02:32 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0EC9CC804AE for ; Wed, 14 Dec 2016 19:02:32 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from weser.webweaving.org (weser.webweaving.org [148.251.234.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.webweaving.org", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A45541EC7 for ; Wed, 14 Dec 2016 19:02:31 +0000 (UTC) (envelope-from dirkx@webweaving.org) Received: from beeb.leiden.webweaving.org (5ED06D14.cm-7-1b.dynamic.ziggo.nl [94.208.109.20]) (authenticated bits=0) by weser.webweaving.org (8.15.2/8.15.2) with ESMTPSA id uBEJ1CcP099747 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 14 Dec 2016 20:01:13 +0100 (CET) (envelope-from dirkx@webweaving.org) X-Authentication-Warning: weser.webweaving.org: Host 5ED06D14.cm-7-1b.dynamic.ziggo.nl [94.208.109.20] claimed to be beeb.leiden.webweaving.org Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Re: ZFS - directory entry From: Dirk-Willem van Gulik In-Reply-To: <20161214185154.GT61036@server.rulingia.com> Date: Wed, 14 Dec 2016 20:01:12 +0100 Cc: FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <68290275-9C8D-4ACC-9E29-B6A7805512FD@webweaving.org> References: <20161214185154.GT61036@server.rulingia.com> To: Peter Jeremy X-Mailer: Apple Mail (2.3259) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (weser.webweaving.org [148.251.234.232]); Wed, 14 Dec 2016 20:01:13 +0100 (CET) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 19:02:32 -0000 > On 14 Dec 2016, at 19:51, Peter Jeremy wrote: >=20 > On 2016-Dec-14 16:27:00 +0100, Dirk-Willem van Gulik = wrote: >> A rather odd directory entry (in /root, the home dir of root/toor) = appeared on a bog standard FreeBSD 10.2 (p18) lightly loaded machine = under ZFS during/post a backup: >>=20 >> $ ls -la /root | tail -q >> ---------- 1 root wheel 9223372036854775807 Jan 1 1970 = ?%+?kD?H???x,?5?Dh;*s!?h???jw??????\h?:????????``?13?@?????OA????????Puux?= ???>=20 >> OS and ZFS is installed with a bog standard sysinstall. =E2=80=98SMART=E2= =80=99 nor smartd have reported anything. nothing in dmesg, syslog of = boot log. Any suggestions as how to debug or get to the root of this ?=20= >>=20 >> And in particular - what is a risk of a reboot (to get a kernel with = debug, etc) causing the issue to =E2=80=98go away=E2=80=99 - and hence = stopping the forensic ? >=20 > Do you have ECC RAM? If not, it's possible this is an artifact of = some RAM > corruption, rather than on-disk corruption. No ECC (simple 4x8 Gbyte sticks; nothing on L2 and L3 either). > I'm surprised by the slow scrub, though they are very slow disks. You = might > like to use gstat or zpool iostat to see if one of the disks is slower = than > the others - indicating a possible problem with it. Perfectly balanced. Rocks steady last few hours according to per disk = graphs. We=E2=80=99ll see how it goes before rebooting (and if it is = indeed memory - that will I guess mask the issue until it appears = again). Thanks, Dw= From owner-freebsd-hackers@freebsd.org Wed Dec 14 19:05:56 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE6CFC8071B for ; Wed, 14 Dec 2016 19:05:56 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id BC6BE300 for ; Wed, 14 Dec 2016 19:05:56 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id B8C4CC8071A; Wed, 14 Dec 2016 19:05:56 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8632C80719 for ; Wed, 14 Dec 2016 19:05:56 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qk0-x231.google.com (mail-qk0-x231.google.com [IPv6:2607:f8b0:400d:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 73A122FF for ; Wed, 14 Dec 2016 19:05:56 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-qk0-x231.google.com with SMTP id n21so31793372qka.3 for ; Wed, 14 Dec 2016 11:05:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=bmTYpi5X129Ty2PwOvcDKZ4AyA1jcef6q6XqOtRXFS8=; b=smzm76x6CMjW5lC9qfAMCwLgJOPBDrtOji4Atb/f6yKGYfN7MmBUCDG8TxJSLvI87/ IQuQ1MoRtgf5LqBe7MkdeyZr7lni/7qJfpAeKjuuDstwqo3olJJ5SaxHzmVRmP0RN05/ ZzUBZzo+Fp8OeUyCppN1xgiYACo4BjTs8kI4+j/H2LyhJWO+jobhmlr9cMk1kKPhsDoF SjffL50rliPzO3L4Eq7WS/aBhMuc9kLVfHq9PNzC0+QBnoaP8nO5Gbx+kRRLYZ0/xgl9 6RaxFuE5ZSCFYX80LMHXZGcl76HQvgQe0CkkMeZfoudo/P46mDzR4WgBp0ofrjtCd+YA eT4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=bmTYpi5X129Ty2PwOvcDKZ4AyA1jcef6q6XqOtRXFS8=; b=gxSOwMiI4siifZiJ2rMuF4SMpGf8B8NarmtueZna1pQcF920DxuzmVZxPSKVDVzKGy TQrkk/Pmd5CcvsTeiRpbLkLgdA8HY4Xv9SacDGnuoxj7Z9F9yQUlP5ShK5lJithzGsh5 ufyMfPAIpElelZEESc3UQQ1i226PyqbZJGDvgpSMMYxUpnZoI5o8hqlxXUH61hMrDrLQ g8HOWlxwh6WnVMYu4R0NGEJtxxoNDmwtbjS3wk3IqmFK1p/5wG40KOC43mA/Cg+9OBcq z+X91zKDM+NRwITy4WLLDEdNDcCd1s+P3ditH1CSxbXirmev+TOuGnkSJu52Qzq/da72 P79w== X-Gm-Message-State: AKaTC02zM392u6amOkCXVkEPDyHhyZ4LylcCvrd5HubfLAWzsq1Zj4KIceCnJU1XmuCzXw3p2uuVnKTz7ugf7w== X-Received: by 10.55.80.198 with SMTP id e189mr39201729qkb.222.1481742355670; Wed, 14 Dec 2016 11:05:55 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.12.174.145 with HTTP; Wed, 14 Dec 2016 11:05:55 -0800 (PST) In-Reply-To: <68290275-9C8D-4ACC-9E29-B6A7805512FD@webweaving.org> References: <20161214185154.GT61036@server.rulingia.com> <68290275-9C8D-4ACC-9E29-B6A7805512FD@webweaving.org> From: Alan Somers Date: Wed, 14 Dec 2016 12:05:55 -0700 X-Google-Sender-Auth: Z1mCm5iDdtSW4AAPNNJzzvLWJl0 Message-ID: Subject: Re: ZFS - directory entry To: Dirk-Willem van Gulik Cc: Peter Jeremy , FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 19:05:57 -0000 On Wed, Dec 14, 2016 at 12:01 PM, Dirk-Willem van Gulik wrote: > >> On 14 Dec 2016, at 19:51, Peter Jeremy wrote: >> >> On 2016-Dec-14 16:27:00 +0100, Dirk-Willem van Gulik wrote: >>> A rather odd directory entry (in /root, the home dir of root/toor) appe= ared on a bog standard FreeBSD 10.2 (p18) lightly loaded machine under ZFS = during/post a backup: >>> >>> $ ls -la /root | tail -q >>> ---------- 1 root wheel 9223372036854775807 Jan 1 1970 ?%+?kD?H??= ?x,?5?Dh;*s!?h???jw??????\h?:????????``?13?@?????OA????????Puux????>> >>> OS and ZFS is installed with a bog standard sysinstall. =E2=80=98SMART= =E2=80=99 nor smartd have reported anything. nothing in dmesg, syslog of bo= ot log. Any suggestions as how to debug or get to the root of this ? >>> >>> And in particular - what is a risk of a reboot (to get a kernel with de= bug, etc) causing the issue to =E2=80=98go away=E2=80=99 - and hence stoppi= ng the forensic ? >> >> Do you have ECC RAM? If not, it's possible this is an artifact of some = RAM >> corruption, rather than on-disk corruption. > > No ECC (simple 4x8 Gbyte sticks; nothing on L2 and L3 either). > >> I'm surprised by the slow scrub, though they are very slow disks. You m= ight >> like to use gstat or zpool iostat to see if one of the disks is slower t= han >> the others - indicating a possible problem with it. > > Perfectly balanced. Rocks steady last few hours according to per disk gra= phs. We=E2=80=99ll see how it goes before rebooting (and if it is indeed me= mory - that will I guess mask the issue until it appears again). > > Thanks, > > Dw If your system is mostly idle, then you can speedup scrub with the following sysctls: vfs.zfs.resilver_min_time_ms=3D5000 vfs.zfs.top_maxinflight=3D1024 # 1024 is appropriate for 8GB of ram. With more RAM, you can go higher. -Alan From owner-freebsd-hackers@freebsd.org Wed Dec 14 20:56:08 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9ECABC77925 for ; Wed, 14 Dec 2016 20:56:08 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 81EEB1EE5 for ; Wed, 14 Dec 2016 20:56:08 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: c07e089e-c23f-11e6-9f67-d3961ed5a660 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id c07e089e-c23f-11e6-9f67-d3961ed5a660; Wed, 14 Dec 2016 20:56:15 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id uBEKu0Ec004438; Wed, 14 Dec 2016 13:56:00 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1481748960.1889.398.camel@freebsd.org> Subject: Re: How to use sem_timedwait? From: Ian Lepore To: Goran =?iso-8859-2?Q?Meki=E6?= , freebsd-hackers@freebsd.org Date: Wed, 14 Dec 2016 13:56:00 -0700 In-Reply-To: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> Content-Type: text/plain; charset="iso-8859-2" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 20:56:08 -0000 On Wed, 2016-12-14 at 08:42 +0100, Goran Mekić wrote: > Hello, > > I'm trying to understand how sem_timedwait() works. For that I > assambled a little program: > > #include > #include > #include > #include > #include > > > int main() { >   sem_t semaphore; >   struct timespec ts; >   clock_gettime(CLOCK_REALTIME, &ts); >   ts.tv_sec += 3; > >   sem_init(&semaphore, 0, 0); >   int result = sem_timedwait(&semaphore, &ts); >   int error = errno; > >   printf("The sem_timedwait() should have ended at %lds and %ldns\n", > ts.tv_sec, ts.tv_nsec); >   clock_gettime(CLOCK_REALTIME, &ts); >   printf("The sem_timedwait() exited at %lds and %ldns with %d > status\n", ts.tv_sec, ts.tv_nsec, result); > >   clock_getres(CLOCK_REALTIME, &ts); >   printf("Clock resolution is %lds and %ldns\n", ts.tv_sec, > ts.tv_nsec); > >   if (result == -1) { >     switch (error) { >     case EINVAL: >       printf("The semaphore does not refer to valid structure\n"); >       break; >     case ETIMEDOUT: >       printf("Timeout occured\n"); >       break; >     case EINTR: >       printf("Interupted\n"); >       break; >     default: >       printf("Unknown error\n"); >       break; >     } >   } > } > > > Running it I get the following output: > > The sem_timedwait() should have ended at 1481700752s and 2050646ns > The sem_timedwait() exited at 1481700752s and 3813780ns with -1 > status > Clock resolution is 0s and 1ns > Timeout occured > > > What I want to know is why the number of nanoseconds reported first > and second time is not the same if the resolution reported on line 3 > is 1ns? It doesn't change much if I use CLOCK_RELATIME_PRECISE. Thank > you! > > Regards, > meka One of the things you did with that code is measured how much time it took to format and print the "It should have ended..." line.  If you want to capture how long you were asleep you need the clock_gettime() to be the next thing you call after sem_timedwait().  Even the time to access errno may be non-trivial if it's actually a thread-local variable. If you want to get better sleep-timing performance (at the expense of less power-saving efficiency), try setting   sysctl kern.timecounter.alloweddeviation=0 That will prevent the aggregation of timeouts scheduled near each other to all happen at the same time, resulting in more wakeups, each of which is more accurate (more wakeups = more power used). -- Ian From owner-freebsd-hackers@freebsd.org Wed Dec 14 23:39:27 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3DBA8C80161 for ; Wed, 14 Dec 2016 23:39:27 +0000 (UTC) (envelope-from mureninc@gmail.com) Received: from mail-vk0-x243.google.com (mail-vk0-x243.google.com [IPv6:2607:f8b0:400c:c05::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DB7EB8C8; Wed, 14 Dec 2016 23:39:26 +0000 (UTC) (envelope-from mureninc@gmail.com) Received: by mail-vk0-x243.google.com with SMTP id w194so5794166vkw.3; Wed, 14 Dec 2016 15:39:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=gTSDIzPteOi9U/aryYrB8aIIEFVl57zfY+HsGd98h3A=; b=SZtexexQS9FoxzQ6YXce2MI/EJNCPDyHY6gQ3rQCXqqIYHCz8qTfp+OxQECTTvNmox iwzL3Y84ehjM2S0G1TGbqJhUK3bbbgjUyruCY/25+139G1dIBHVchV1pgh5tEnk+vEbu I5QLmVir0TEx5hWlmhOcs73REOVXF4vpj56sdqJZrwyVlvTKhDQ0DogpzqAGffY69Rkp Osy31UwNsg9yrGr1ixKEapVE6K053f47p/3Q/WmtuH7H98K35KjKY0RduisSqu6BCrOJ DdcYDYQHXop7BrU7MYxPBk3Xjiv0MJrd7kSQ48RHPcZD+iJZJANTb0vEyEmpTaGzOVji +5vA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=gTSDIzPteOi9U/aryYrB8aIIEFVl57zfY+HsGd98h3A=; b=srNb8cPMyN6VRLRrDwg9qRuZBWxHpePa7IBSuaDOzkJ0giTf3GQ33sl6mQBxsbRiFj YFo/Nqe05yMcEh01rtyAIWxv8vgz0tmleMfRM65WrngxarHl7gqkBbBmN5l4A/Xj0grd BBKUP98KDfz9obGJKYXxQAPOGBujcgtZOmNDerxSx6wRjKi8eOW6jWI0scjrTS0ZDsQz UKpxisImf7k2egJw1EJRAsGLsai4yuz3lRmg7VYSdJ0xtfMg6vN6xHGlgJc8wnziDf6A uqSdKXHjcLnFiNQcylFliThYHkpTqKkZYYmQuZTaHOTA/9hR02mPxEPQtDf9cVf3o7cp fk0Q== X-Gm-Message-State: AKaTC01aYgQnoi9lKUPDnXFRhK2rhmvxrHlJDSai3zL4GalbKOOv96Suwp+g6fWJSjT+YW7qXd0a1VmG8zhyCQ== X-Received: by 10.176.82.48 with SMTP id i45mr75193124uaa.126.1481758765801; Wed, 14 Dec 2016 15:39:25 -0800 (PST) MIME-Version: 1.0 Sender: mureninc@gmail.com Received: by 10.159.36.16 with HTTP; Wed, 14 Dec 2016 15:39:24 -0800 (PST) In-Reply-To: <20161214142048.Horde.JOZn2cO5FyhzaoCSGj30FT1@webmail.leidinger.net> References: <200A5D70-24CE-4843-B99D-E10A2AF4F68F@FreeBSD.org> <20161213052759.GP61036@server.rulingia.com> <20161214142048.Horde.JOZn2cO5FyhzaoCSGj30FT1@webmail.leidinger.net> From: "Constantine A. Murenin" Date: Wed, 14 Dec 2016 17:39:24 -0600 X-Google-Sender-Auth: ZTj7DCwOy3nHONUh7vx-mf4z3tI Message-ID: Subject: Re: Sysctl as a Service, or: making sysctl(3) more friendly for monitoring systems To: Alexander Leidinger , "M. Warner Losh" Cc: Adrian Chadd , FreeBSD Hackers , Ed Schouten , "Constantine A. Murenin" , Andriy Gapon Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 23:39:27 -0000 On 14 December 2016 at 07:20, Alexander Leidinger wrote: > Quoting Adrian Chadd (from Mon, 12 Dec 2016 > 23:32:00 -0800): > >> On 12 December 2016 at 22:03, Warner Losh wrote: >>> >>> On Mon, Dec 12, 2016 at 10:49 PM, Adrian Chadd >>> wrote: >>>> >>>> If only we had a mechanism for these kinds of sensors to register, so >>>> the API they'd be using would be one that lead itself to edge/level >>>> triggered comparisons, versus polling. >>>> >>>> Wait, didn't we /have/ one of those up for review a few years ago, and >>>> it wasn't merged in? >>> >>> >>> Didn't we have people willing to work with the folks that put this up >>> when to address the defects in the review? Oh, wait, they never wanted >>> to work to fix the problems, even with many offers of help to do so. Most of the discussion was heavily politicised. It was a long time ago; however, I do not recall anyone making any concrete offers of what exactly had to be fixed. I think it would only be fair to say that someone didn't want "to work to fix the problems" if by that you mean that after so much backlash/politics, someone wasn't motivated enough to start an entirely new project, entirely from scratch, on their own spare time, even though an existing one did the job already. In fact, this wasn't just my own conclusion, many others have arrived at the same conclusion as well. That is not to say that the framework is without issues, however. I've documented my view on the subject within my MMath CS thesis, you can jump directly to Section 8, "Port to FreeBSD / DragonFly BSD": http://uwspace.uwaterloo.ca/handle/10012/5234 http://uwspace.uwaterloo.ca/bitstream/handle/10012/5234/UW.MMath.CS.mur= enin.cnst-sensors.2010-05-21.pages09.pdf >>> >>> My offer still stands, btw. As it's been almost ten years, perhaps you are thinking of another, more recent discussion instead: http://lists.freebsd.org/pipermail/freebsd-acpi/2010-April/006452.html Your comments and guidance in that thread was very specific, helpful and valued, got immediately addressed to everyone's mutual satisfaction, and the results were successfully committed without any controversy, 6 years and counting =E2=80=94 the *BSD aibs(4) driver in question is still present in FreeBSD, as well as all the other BSDs, too: http://mdoc.su/f/aibs.4 http://mdoc.su/f110,n70,d,o60/aibs.4 If there are any concrete dealbreaker issues that could likewise be addressed w/r/t the sensors framework, I'd be happy to hear them. >> >> >> Heh, this wasn't a troll? :) I should've been clearer! >> >> Does anyone remember where the ye olde framework was? > > > Original: > - Google Summer of Code 2007. > - I don't find the repo in svnweb right now > (https://svnweb.freebsd.org/socsvn/). Where do we have the things from > before 2011? We've had p4 back in the day; everything is still there, even p4web is still up, summary and the links are at: http://wiki.freebsd.org/GSoC2007/cnst-sensors Easiest way to bring back the code would probably be to simply pick up the 3 commits from CVS src HEAD / SVN base/head: http://svnweb.freebsd.org/base?view=3Drevision&revision=3D172631 http://svnweb.freebsd.org/base?view=3Drevision&revision=3D172632 http://svnweb.freebsd.org/base?view=3Drevision&revision=3D172633 History and background are available in my thesis: http://cnst.su/MMathCS http://uwspace.uwaterloo.ca/handle/10012/5234 > > There was at least in the past someone keeping it at least up-to-date, bu= t I > don't remember right now who/where. IIRC, there were a number of people that tried maintaining it henceforth, both publicly and privately, and the only issue you'd have applying r172631 to -CURRENT would probably be a few simple merge conflicts (Makefile etc), very simple to resolve. There was a public thread from 2012 by avg@: http://lists.freebsd.org/pipermail/freebsd-arch/2012-October/013180.htm= l > > If someone is interested in the mail-thread of the past... available on > request. Actually, I think an mbox file is already publicly available from here: http://www.leidinger.net/blog/2009/12/06/freenas-sensors-for-freebsd :-) Cheers, Constantine.SU. http://Constantine.SU/ > > Bye, > Alexander. > -- > http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF > http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF From owner-freebsd-hackers@freebsd.org Thu Dec 15 00:29:39 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7166DC80D24 for ; Thu, 15 Dec 2016 00:29:39 +0000 (UTC) (envelope-from meka@tilda.center) Received: from mail.tilda.center (mail.tilda.center [188.226.130.133]) by mx1.freebsd.org (Postfix) with ESMTP id 2F8EB1DD3; Thu, 15 Dec 2016 00:29:38 +0000 (UTC) (envelope-from meka@tilda.center) Received: from hal9000.meka.no-ip.org (unknown [87.116.181.51]) by mail.tilda.center (Postfix) with ESMTPSA id 2A98E61551; Thu, 15 Dec 2016 01:29:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=tilda.center; s=mail; t=1481761746; bh=3doPPtODrFPljFBh9Hpz5gr9Kx+0WEE65ciMFSf+klc=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=U8d+calvSTVQ4QmhR2xX+S3u7pIrVhJJ8+rscEC4WOR5/aTVnnZCrGMFQ4/ndSDGR +68mFpmnVyOPXckZ8OV2bS+EogJsDr0xT6xml6IRZo5kt5frS9WJNyvKIOsMJnOQr2 iaJVQv5M1Qc9I1bi6DDxrcYGV1vCR6wRLt5njTTk= Date: Thu, 15 Dec 2016 01:29:06 +0100 From: Goran =?utf-8?B?TWVracSH?= To: Ian Lepore Cc: freebsd-hackers@freebsd.org Subject: Re: How to use sem_timedwait? Message-ID: <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="hrhuntinyslroab3" Content-Disposition: inline In-Reply-To: <1481748960.1889.398.camel@freebsd.org> User-Agent: NeoMutt/20161126 (1.7.1) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 00:29:39 -0000 --hrhuntinyslroab3 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Dec 14, 2016 at 01:56:00PM -0700, Ian Lepore wrote: > On Wed, 2016-12-14 at 08:42 +0100, Goran Meki=C4=87 wrote: > One of the things you did with that code is measured how much time it > took to format and print the "It should have ended..." line. =C2=A0If you > want to capture how long you were asleep you need the clock_gettime() > to be the next thing you call after sem_timedwait(). =C2=A0Even the time = to > access errno may be non-trivial if it's actually a thread-local > variable. > > If you want to get better sleep-timing performance (at the expense of > less power-saving efficiency), try setting > > =C2=A0 sysctl kern.timecounter.alloweddeviation=3D0 > > That will prevent the aggregation of timeouts scheduled near each other > to all happen at the same time, resulting in more wakeups, each of > which is more accurate (more wakeups =3D more power used). > > -- Ian > This is exactly what I've needed. Thank you so much. If you can tell me any= other tip to make the difference even smaller, I'd be grateful. Currently,= it goes from 0.1ms to 1ms. Great improvement! Once more, thank you very mu= ch! Regards, meka --hrhuntinyslroab3 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE1WIFkXy2ZeMKjjKEWj1TknovrLYFAlhR488ACgkQWj1Tknov rLbjixAAnqaNtBOX9tv04Udv5UX6gN947P2fSOnTz0/VBKl2Mlu24uRtJ3AL/Bhl VWn2mJiS4zR/ty/S4bajCso+rJwjTEeo9PJ7TGA9j0RmSXWht4K+V2lJH0dZKWCD FOKAJWkZVQugcv+ESzw3AXmzGe8mXoyivcPAa9Wr+Irt8U7U+9OckhrOTTghJ9EL n+PrtYf9DKpsugjaJ5NWKrTdASIYOyjIxXuUaW5hdHV5f5+Q5ZOx4PK8XJkM5YvK 2d9/HZfFRyFby+8+Exy6/9cTkaxaQgTNjcKhXgCOq7rn1gYxfbKol8WyG9LQmF24 cibW+frmX9Xi2uvB4E9Sr8qs3TPz2hhfD0Qgfnhlbmk3th9q7gmk++kf+tPDoJwG QzjA0qaJufrFEwVz7/faZb2S8wuGr8U/TMTwXPlMOAmAdxcHq0iT7GpTpua6ix11 MKi+s7FWSvr0r5n0QZvuLFoLK9y7h+Bl4lAfdpDopc1V4NCkeF+/A1LmqBDVc2uR TG4IAfmw7OY4A+tb7sYnXboQBOH3k7bdBUJD31TTEq+WveOdRAzazAjY88sGVt2p ZxlJ6ZPlRchxMmncRqh4HsM0aa+UYCRdwJElmvfOPwtj/X4xcjVCvKRolPpjRL7Y lJTw6WHukvqAf0Fwi7mq7Cpb65oZRC6kPM3OJ/fmN7cFyelO5m4= =GzqM -----END PGP SIGNATURE----- --hrhuntinyslroab3-- From owner-freebsd-hackers@freebsd.org Thu Dec 15 04:18:34 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E0C44C81670 for ; Thu, 15 Dec 2016 04:18:34 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C5DDFC6B for ; Thu, 15 Dec 2016 04:18:34 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: 9267135b-c27d-11e6-9f67-d3961ed5a660 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id 9267135b-c27d-11e6-9f67-d3961ed5a660; Thu, 15 Dec 2016 04:18:46 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id uBF4IV15005076; Wed, 14 Dec 2016 21:18:31 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1481775511.1889.450.camel@freebsd.org> Subject: Re: How to use sem_timedwait? From: Ian Lepore To: Goran =?iso-8859-2?Q?Meki=E6?= Cc: freebsd-hackers@freebsd.org Date: Wed, 14 Dec 2016 21:18:31 -0700 In-Reply-To: <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> Content-Type: text/plain; charset="iso-8859-2" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 04:18:35 -0000 On Thu, 2016-12-15 at 01:29 +0100, Goran Mekić wrote: > On Wed, Dec 14, 2016 at 01:56:00PM -0700, Ian Lepore wrote: > > > > On Wed, 2016-12-14 at 08:42 +0100, Goran Mekić wrote: > > One of the things you did with that code is measured how much time > > it > > took to format and print the "It should have ended..." line.  If > > you > > want to capture how long you were asleep you need the > > clock_gettime() > > to be the next thing you call after sem_timedwait().  Even the time > > to > > access errno may be non-trivial if it's actually a thread-local > > variable. > > > > If you want to get better sleep-timing performance (at the expense > > of > > less power-saving efficiency), try setting > > > >   sysctl kern.timecounter.alloweddeviation=0 > > > > That will prevent the aggregation of timeouts scheduled near each > > other > > to all happen at the same time, resulting in more wakeups, each of > > which is more accurate (more wakeups = more power used). > > > > -- Ian > > > This is exactly what I've needed. Thank you so much. If you can tell > me any other tip to make the difference even smaller, I'd be > grateful. Currently, it goes from 0.1ms to 1ms. Great improvement! > Once more, thank you very much! > > Regards, > meka Do you actually need semaphore behavior along with the precise sleeping, or are you just using it because it is a convenient function that takes a timespec/nanosecs wait time? Making a guess here:  Is your actual goal to wake up periodically with the period between wakeups as accurate as possible?  If so, a better mechanism might be to use kqueue(2)/kevent(2) with EVFILT_TIMER events.  With EVFILT_TIMER events, the wakeups are scheduled as if aligned to a grid -- even if one wakeup is a bit late due to system workload, the next wakeup after it will still be properly aligned to the original grid.  For example, if you ask for a wakeup once per millisecond and some wake happens after 1.2mS, the next wakeup will be .8mS after that; the phase of the wakeups doesn't shift over time. -- Ian From owner-freebsd-hackers@freebsd.org Thu Dec 15 04:35:45 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6C3E9C81C2F for ; Thu, 15 Dec 2016 04:35:45 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1a.eu.mailhop.org (outbound1a.eu.mailhop.org [52.58.109.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0A25E166C for ; Thu, 15 Dec 2016 04:35:44 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: eddc8612-c27f-11e6-9673-39b5816e8152 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.eu.mailhop.org (Halon) with ESMTPSA id eddc8612-c27f-11e6-9673-39b5816e8152; Thu, 15 Dec 2016 04:35:40 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id uBF4ZWQq005095; Wed, 14 Dec 2016 21:35:32 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1481776532.1889.461.camel@freebsd.org> Subject: Re: How to use sem_timedwait? From: Ian Lepore To: Goran =?iso-8859-2?Q?Meki=E6?= Cc: freebsd-hackers@freebsd.org Date: Wed, 14 Dec 2016 21:35:32 -0700 In-Reply-To: <1481775511.1889.450.camel@freebsd.org> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> Content-Type: text/plain; charset="iso-8859-2" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 04:35:45 -0000 On Wed, 2016-12-14 at 21:18 -0700, Ian Lepore wrote: > On Thu, 2016-12-15 at 01:29 +0100, Goran Mekić wrote: > > > > On Wed, Dec 14, 2016 at 01:56:00PM -0700, Ian Lepore wrote: > > > [...] >  > Making a guess here:  Is your actual goal to wake up periodically > with the period between wakeups as accurate as possible?  If so, a > better mechanism might be to use kqueue(2)/kevent(2) > with EVFILT_TIMER events.  With EVFILT_TIMER events, the wakeups are > scheduled as if aligned to a grid -- even if one wakeup is a bit late > due to system workload, the next wakeup after it will still be > properly aligned to the original grid.  For example, if you ask for a > wakeup once per millisecond and some wake happens after 1.2mS, the > next wakeup will be .8mS after that; the phase of the wakeups doesn't > shift over time. I just dug up some old source code for testing kevent timers, included below.  Good news and bad news...  The good news is that it works perfectly on an arm system running -current: root@imx6 :~ # ./kevent_evfilt_timer nsec = 10000058742 for 10000 loops of 1000000 nsec The extra 58uS is fixed overhead for getting the ending time; if the loop runs for 100 seconds instead of 10 the extra time is still 55- 60uS. The bad news is that it doesn't work right on amd64 running 10-stable: revolution > ./kevent_evfilt_timer  nsec = 9313236220 for 10000 loops of 1000000 nsec I don't have any other x86 systems handy to test it on right now, but it's disturbing that 10 seconds worth of 1mS sleeps takes less than 10 seconds.  A strange thing here is that the *ratio* of the undersleeping is fixed, running the loop for 100 seconds instead of 10 gives: revolution > ./kevent_evfilt_timer  nsec = 93132267106 for 100000 loops of 1000000 nsec The program.... /*  * This tests kevent timer events to make sure that they are properly  * metronomic.  That is, each event should occur at the requested  * interval following the prior event, and in particular the phase of  * event delivery should not drift because of the time it takes to  * process the event and schedule the next one in the kernel.  (i.e.,  * this is a userland test of kernel kevent timer performance.)  * Because it's hard to measure the tiny increments of time between  * each sleep, we run a loop that takes several seconds and look for  * the total elapsed time to be increment*loopcount plus a tiny bit of  * overhead for getting the ending time.  */ #include #include #include #include #include #include #include #include int main(int argc, char ** argv) {     int i, interval;     int64_t nsec;     int kq,num_events;     struct kevent inqueue;     struct kevent outqueue;     struct timespec start,end;     // argument is timer interval in nanoseconds.     if (argc > 1)         interval = strtoul(argv[1], NULL, 0);     else         interval = 1000000;     if ((kq = kqueue()) == -1) {         fprintf(stderr, "kqueue errno = %s", strerror(errno));         exit(EXIT_FAILURE);     }     EV_SET(&inqueue, 1, EVFILT_TIMER, EV_ADD | EV_ENABLE,          NOTE_NSECONDS, interval, 0);     clock_gettime(CLOCK_MONOTONIC_PRECISE, &start);     for (i = 0; i < 10000000000 / interval; i++) {         num_events = kevent(kq, &inqueue, 1, &outqueue, 1, NULL);         if (num_events == -1) {             fprintf(stderr, "kevent errno = %s", strerror(errno));             exit(EXIT_FAILURE);         } else if (outqueue.flags & EV_ERROR) {             fprintf(stderr, "EV_ERROR: %s\n", strerror(outqueue.data));             exit(EXIT_FAILURE);         }         if (num_events != 1)             printf("num_events %d at i %d\n", num_events, i);     }     clock_gettime(CLOCK_MONOTONIC_PRECISE, &end);     nsec = (end.tv_sec * 1000000000LL + end.tv_nsec) -          (start.tv_sec * 1000000000LL + start.tv_nsec);     printf("nsec = %jd for %d loops of %d nsec\n",          (intmax_t)nsec, i, interval);     close(kq);     return EXIT_SUCCESS; } -- Ian From owner-freebsd-hackers@freebsd.org Thu Dec 15 08:20:20 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 19C4EC81E83 for ; Thu, 15 Dec 2016 08:20:20 +0000 (UTC) (envelope-from mj@bsdops.com) Received: from da.strony-domeny.pl (da.strony-domeny.pl [178.32.204.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D698FDFC for ; Thu, 15 Dec 2016 08:20:19 +0000 (UTC) (envelope-from mj@bsdops.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bsdops.com; s=x; h=Message-ID:Reply-To:Subject:To:From:Date:Content-Transfer-Encoding: Content-Type:MIME-Version; bh=8mB2uhiU9i/XunTJqHPrPZeKEorX3d0wc0nO+p/G2Lg=; b=m6Vjktrnet0ildtD6hoCSKDa8CGrq9AOeXnVkvYn3R8NjfelQl170XMNE2bTWcuU8CqwG0qpSh jgpUKoPTJA5txVrnWKoxlZas8ZWM8R950wJm2GtdSQP5MPdp5MwM1aehdmvS3MjgX29FzFbJfpI0S ccyuzM8Vxhq6hZLvxj0TShMP91QcqKzNPa8I+kaE08cDc46Ix4hA7eaTh6aweChQ974dNpM7HJl1w 6zi7OfvuRCOdZBuW4aCzAqMUne2zk96e/4MFMAI3uCBHEgvtosBagiiTszS5DLBEc1ev1PDko2NtN erjD7PYxvRx0hp4n1uOg5YImYnaBmjs+KZFuQ==; Received: from [127.0.0.1] (helo=da.strony-domeny.pl) by da.strony-domeny.pl with esmtpa (Exim 4.86) (envelope-from ) id 1cHRGy-000KON-EH for freebsd-hackers@freebsd.org; Thu, 15 Dec 2016 09:20:16 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 15 Dec 2016 09:20:08 +0100 From: =?UTF-8?Q?Micha=C5=82_J=C4=99drzejczak?= To: freebsd-hackers@freebsd.org Subject: gdb 32bit app under 64bit system Organization: BSDops.com Reply-To: mj@bsdops.com Mail-Reply-To: mj@bsdops.com Message-ID: <4b6a32cd221a310de2e10cd18cb1b9cf@bsdops.com> X-Sender: mj@bsdops.com User-Agent: Roundcube Webmail/1.2.3 X-Authenticated-Id: mj@bsdops.com X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 08:20:20 -0000 Hello list. I've question about debugging an application that is compile with -m32 and can not be debugging under FreeBSD 10.0 amd64 with gdb761. During debug by gdb I had this error : warning: `/libexec/ld-elf.so.1': Shared library architecture i386:x86-64 is not compatible with target architecture i386. warning: .dynamic section for "/libexec/ld-elf.so.1" is not at the expected address (wrong library or version mismatch?) [New Thread 298030b429803080 (LWP 100109)] Error while reading shared library symbols for /usr/lib32/libthr.so.3: Cannot find new threads: generic error Did you have any ideas what way can I go . Regards MJ From owner-freebsd-hackers@freebsd.org Thu Dec 15 11:20:58 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0217BC810B2 for ; Thu, 15 Dec 2016 11:20:58 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9B9D21ADF; Thu, 15 Dec 2016 11:20:57 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBFBKpZG064988 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 15 Dec 2016 13:20:51 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBFBKpZG064988 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBFBKo3D064983; Thu, 15 Dec 2016 13:20:50 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 15 Dec 2016 13:20:50 +0200 From: Konstantin Belousov To: Ian Lepore Cc: Goran Meki? , freebsd-hackers@freebsd.org Subject: Re: How to use sem_timedwait? Message-ID: <20161215112050.GO94325@kib.kiev.ua> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> <1481776532.1889.461.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1481776532.1889.461.camel@freebsd.org> User-Agent: Mutt/1.7.2 (2016-11-26) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 11:20:58 -0000 On Wed, Dec 14, 2016 at 09:35:32PM -0700, Ian Lepore wrote: > On Wed, 2016-12-14 at 21:18 -0700, Ian Lepore wrote: > > On Thu, 2016-12-15 at 01:29 +0100, Goran Meki? wrote: > > > > > > On Wed, Dec 14, 2016 at 01:56:00PM -0700, Ian Lepore wrote: > > > > [...] > >š > > Making a guess here: šIs your actual goal to wake up periodically > > with the period between wakeups as accurate as possible? šIf so, a > > better mechanism might be to use kqueue(2)/kevent(2) > > withšEVFILT_TIMER events. šWith EVFILT_TIMER events, the wakeups are > > scheduled as if aligned to a grid -- even if one wakeup is a bit late > > due to system workload, the next wakeup after it will still be > > properly aligned to the original grid. šFor example, if you ask for a > > wakeup once per millisecond and some wake happens after 1.2mS, the > > next wakeup will be .8mS after that; the phase of the wakeups doesn't > > shift over time. Well, this can be alternatively explained as the statement that the timeouts are scheduled to execute at the absolute times, instead of specifying offsets from the current moment. With such formulation, it is obvious that userspace may do the same, calculating the next absolute time based on previous desired moment + offset, instead of current time + offset. sem_timedwait(2) requests timeouts in CLOCK_REALTIME base. Of course, kqueue(2) timers resets are less expensive since all this happens in the callout callback in kernel. > > I just dug up some old source code for testing kevent timers, included > below. šGood news and bad news... šThe good news is that it works > perfectly on an arm system running -current: > > root@imx6 :~ # ./kevent_evfilt_timer > nsec = 10000058742 for 10000 loops of 1000000 nsec > > The extra 58uS is fixed overhead for getting the ending time; if the > loop runs for 100 seconds instead of 10 the extra time is still 55- > 60uS. > > The bad news is that it doesn't work right on amd64 running 10-stable: > > revolution > ./kevent_evfilt_timerš > nsec = 9313236220 for 10000 loops of 1000000 nsec > > I don't have any other x86 systems handy to test it on right now, but > it's disturbing that 10 seconds worth of 1mS sleeps takes less than 10 > seconds. šA strange thing here is that the *ratio* of the undersleeping > is fixed, running the loop for 100 seconds instead of 10 gives: > > revolution > ./kevent_evfilt_timerš > nsec = 93132267106 for 100000 loops of 1000000 nsec What are timecounter and eventcounter used on this machine ? Also please show first 20 lines of the dmesg, where CPU identification is performed. I checked on two machines, one running 11 and using HPET for events: nsec = 10000091075 for 10000 loops of 1000000 nsec another running HEAD and using LAPIC deadline: nsec = 10000129629 for 10000 loops of 1000000 nsec From owner-freebsd-hackers@freebsd.org Thu Dec 15 15:55:52 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E7A0C813D8 for ; Thu, 15 Dec 2016 15:55:52 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1a.eu.mailhop.org (outbound1a.eu.mailhop.org [52.58.109.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BE3B0F9F for ; Thu, 15 Dec 2016 15:55:51 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: f36fe890-c2de-11e6-9673-39b5816e8152 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.eu.mailhop.org (Halon) with ESMTPSA id f36fe890-c2de-11e6-9673-39b5816e8152; Thu, 15 Dec 2016 15:55:51 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id uBFFthQZ001324; Thu, 15 Dec 2016 08:55:43 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1481817343.1889.467.camel@freebsd.org> Subject: Re: How to use sem_timedwait? From: Ian Lepore To: Konstantin Belousov Cc: Goran Meki? , freebsd-hackers@freebsd.org Date: Thu, 15 Dec 2016 08:55:43 -0700 In-Reply-To: <20161215112050.GO94325@kib.kiev.ua> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> <1481776532.1889.461.camel@freebsd.org> <20161215112050.GO94325@kib.kiev.ua> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 15:55:52 -0000 On Thu, 2016-12-15 at 13:20 +0200, Konstantin Belousov wrote: > On Wed, Dec 14, 2016 at 09:35:32PM -0700, Ian Lepore wrote: > > > > On Wed, 2016-12-14 at 21:18 -0700, Ian Lepore wrote: > > > > > > On Thu, 2016-12-15 at 01:29 +0100, Goran Meki? wrote: > > > > > > > > > > > > On Wed, Dec 14, 2016 at 01:56:00PM -0700, Ian Lepore wrote: > > > > > > > > > > [...] > > >   > > > Making a guess here:  Is your actual goal to wake up periodically > > > with the period between wakeups as accurate as possible?  If so, > > > a > > > better mechanism might be to use kqueue(2)/kevent(2) > > > with EVFILT_TIMER events.  With EVFILT_TIMER events, the wakeups > > > are > > > scheduled as if aligned to a grid -- even if one wakeup is a bit > > > late > > > due to system workload, the next wakeup after it will still be > > > properly aligned to the original grid.  For example, if you ask > > > for a > > > wakeup once per millisecond and some wake happens after 1.2mS, > > > the > > > next wakeup will be .8mS after that; the phase of the wakeups > > > doesn't > > > shift over time. > Well, this can be alternatively explained as the statement that the > timeouts are scheduled to execute at the absolute times, instead of > specifying offsets from the current moment. With such formulation, it > is > obvious that userspace may do the same, calculating the next absolute > time based on previous desired moment + offset, instead of current > time > + offset. sem_timedwait(2) requests timeouts in CLOCK_REALTIME base. > > Of course, kqueue(2) timers resets are less expensive since all this > happens > in the callout callback in kernel. > > > > > > > I just dug up some old source code for testing kevent timers, > > included > > below.  Good news and bad news...  The good news is that it works > > perfectly on an arm system running -current: > > > >     root@imx6    :~ # ./kevent_evfilt_timer > >     nsec = 10000058742 for 10000 loops of 1000000 nsec > > > > The extra 58uS is fixed overhead for getting the ending time; if > > the > > loop runs for 100 seconds instead of 10 the extra time is still 55- > > 60uS. > > > > The bad news is that it doesn't work right on amd64 running 10- > > stable: > > > >     revolution > ./kevent_evfilt_timer  > >     nsec = 9313236220 for 10000 loops of 1000000 nsec > > > > I don't have any other x86 systems handy to test it on right now, > > but > > it's disturbing that 10 seconds worth of 1mS sleeps takes less than > > 10 > > seconds.  A strange thing here is that the *ratio* of the > > undersleeping > > is fixed, running the loop for 100 seconds instead of 10 gives: > > > >     revolution > ./kevent_evfilt_timer  > >     nsec = 93132267106 for 100000 loops of 1000000 nsec > What are timecounter and eventcounter used on this machine ? > Also please show first 20 lines of the dmesg, where CPU > identification > is performed. > > I checked on two machines, one running 11 and using HPET for events: > nsec = 10000091075 for 10000 loops of 1000000 nsec > another running HEAD and using LAPIC deadline: > nsec = 10000129629 for 10000 loops of 1000000 nsec It makes me feel a bit better to hear that it's only this machine, which is pretty old.  I wonder if it has something to do with overclocking the cpu?  I never have any other kind of timing trouble, though.  (Given that I work in the field of precision timing, I'm a bit sensitive to such things.) revolution > sysctl kern.timecounter kern.eventtimer kern.timecounter.tsc_shift: 1 kern.timecounter.smp_tsc_adjust: 0 kern.timecounter.smp_tsc: 1 kern.timecounter.invariant_tsc: 1 kern.timecounter.fast_gettime: 1 kern.timecounter.tick: 1 kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) HPET(950) i8254(0) dummy(-1000000) kern.timecounter.hardware: TSC-low kern.timecounter.alloweddeviation: 0 kern.timecounter.stepwarnings: 0 kern.timecounter.tc.TSC-low.quality: 1000 kern.timecounter.tc.TSC-low.frequency: 2125044382 kern.timecounter.tc.TSC-low.counter: 1611837909 kern.timecounter.tc.TSC-low.mask: 4294967295 kern.timecounter.tc.ACPI-fast.quality: 900 kern.timecounter.tc.ACPI-fast.frequency: 3579545 kern.timecounter.tc.ACPI-fast.counter: 6548232 kern.timecounter.tc.ACPI-fast.mask: 16777215 kern.timecounter.tc.HPET.quality: 950 kern.timecounter.tc.HPET.frequency: 14318180 kern.timecounter.tc.HPET.counter: 3623382209 kern.timecounter.tc.HPET.mask: 4294967295 kern.timecounter.tc.i8254.quality: 0 kern.timecounter.tc.i8254.frequency: 1193182 kern.timecounter.tc.i8254.counter: 18218 kern.timecounter.tc.i8254.mask: 65535 kern.eventtimer.periodic: 0 kern.eventtimer.timer: LAPIC kern.eventtimer.idletick: 0 kern.eventtimer.singlemul: 2 kern.eventtimer.choice: LAPIC(600) HPET(350) HPET1(340) HPET2(340) HPET3(340) i8254(100) RTC(0) kern.eventtimer.et.HPET3.quality: 340 kern.eventtimer.et.HPET3.frequency: 14318180 kern.eventtimer.et.HPET3.flags: 3 kern.eventtimer.et.HPET2.quality: 340 kern.eventtimer.et.HPET2.frequency: 14318180 kern.eventtimer.et.HPET2.flags: 3 kern.eventtimer.et.HPET1.quality: 340 kern.eventtimer.et.HPET1.frequency: 14318180 kern.eventtimer.et.HPET1.flags: 3 kern.eventtimer.et.HPET.quality: 350 kern.eventtimer.et.HPET.frequency: 14318180 kern.eventtimer.et.HPET.flags: 3 kern.eventtimer.et.RTC.quality: 0 kern.eventtimer.et.RTC.frequency: 32768 kern.eventtimer.et.RTC.flags: 17 kern.eventtimer.et.i8254.quality: 100 kern.eventtimer.et.i8254.frequency: 1193182 kern.eventtimer.et.i8254.flags: 1 kern.eventtimer.et.LAPIC.quality: 600 kern.eventtimer.et.LAPIC.frequency: 85001804 kern.eventtimer.et.LAPIC.flags: 7 FreeBSD 10.3-STABLE #5 r308807: Fri Nov 18 09:51:39 MST 2016     ilepore@revolution.hippie.lan:/b/staging/machines/revolution10/obj/b/staging/machines/revolution10/src/sys/REVOLUTION10 amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 VT(vga): text 80x25 CPU: Intel(R) Xeon(R) CPU           W3680  @ 3.33GHz (4250.09-MHz K8-class CPU)   Origin="GenuineIntel"  Id=0x206c2  Family=0x6  Model=0x2c  Stepping=2   Features=0xbfebfbff   Features2=0x29ee3ff   AMD Features=0x2c100800   AMD Features2=0x1   VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID   TSC: P-state invariant, performance statistics real memory  = 12884901888 (12288 MB) avail memory = 12425519104 (11849 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <092410 APIC1941> FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 SMT threads -- Ian From owner-freebsd-hackers@freebsd.org Thu Dec 15 16:47:34 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 64AB6C81857 for ; Thu, 15 Dec 2016 16:47:34 +0000 (UTC) (envelope-from lidl@pix.net) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254:11::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 406B01925 for ; Thu, 15 Dec 2016 16:47:34 +0000 (UTC) (envelope-from lidl@pix.net) Received: from torb.pix.net (torb.pix.net [192.168.16.32]) (authenticated bits=0) by hydra.pix.net (8.16.0.19/8.15.2) with ESMTPA id uBFGlX9U077559; Thu, 15 Dec 2016 11:47:33 -0500 (EST) (envelope-from lidl@pix.net) Subject: Re: How to use sem_timedwait? To: freebsd-hackers@freebsd.org References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> <1481776532.1889.461.camel@freebsd.org> From: Kurt Lidl Message-ID: <44ad49f5-75ec-7c1d-28dc-25df08c67148@pix.net> Date: Thu, 15 Dec 2016 11:47:33 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <1481776532.1889.461.camel@freebsd.org> Content-Type: text/plain; charset=iso-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 16:47:34 -0000 On 12/14/16 11:35 PM, Ian Lepore wrote: > /* > * This tests kevent timer events to make sure that they are properly > * metronomic. That is, each event should occur at the requested > * interval following the prior event, and in particular the phase of > * event delivery should not drift because of the time it takes to > * process the event and schedule the next one in the kernel. (i.e., > * this is a userland test of kernel kevent timer performance.) > * Because it's hard to measure the tiny increments of time between > * each sleep, we run a loop that takes several seconds and look for > * the total elapsed time to be increment*loopcount plus a tiny bit of > * overhead for getting the ending time. > */ > > #include > #include > #include > #include > #include > #include > #include > #include > > int > main(int argc, char ** argv) > { > int i, interval; > int64_t nsec; > int kq,num_events; > struct kevent inqueue; > struct kevent outqueue; > struct timespec start,end; > > // argument is timer interval in nanoseconds. > if (argc > 1) > interval = strtoul(argv[1], NULL, 0); > else > interval = 1000000; > > if ((kq = kqueue()) == -1) { > fprintf(stderr, "kqueue errno = %s", strerror(errno)); > exit(EXIT_FAILURE); > } > EV_SET(&inqueue, 1, EVFILT_TIMER, EV_ADD | EV_ENABLE, > NOTE_NSECONDS, interval, 0); > > clock_gettime(CLOCK_MONOTONIC_PRECISE, &start); > > for (i = 0; i < 10000000000 / interval; i++) { > num_events = kevent(kq, &inqueue, 1, &outqueue, 1, NULL); > if (num_events == -1) { > fprintf(stderr, "kevent errno = %s", strerror(errno)); > exit(EXIT_FAILURE); > } else if (outqueue.flags & EV_ERROR) { > fprintf(stderr, "EV_ERROR: %s\n", strerror(outqueue.data)); > exit(EXIT_FAILURE); > } > if (num_events != 1) > printf("num_events %d at i %d\n", num_events, i); > } > clock_gettime(CLOCK_MONOTONIC_PRECISE, &end); > > nsec = (end.tv_sec * 1000000000LL + end.tv_nsec) - > (start.tv_sec * 1000000000LL + start.tv_nsec); > printf("nsec = %jd for %d loops of %d nsec\n", > (intmax_t)nsec, i, interval); > > close(kq); > return EXIT_SUCCESS; > } When I run this code on a sparc64, it panics the machine! lidl@ton-148: ./x panic: trap: fast data access mmu miss (kernel) KDB: stack backtrace: vpanic() at vpanic+0xfc panic() at panic+0x20 trap() at trap+0x554 -- fast data access mmu miss tar=0xc09eda8f %o7=0xc039b344 -- userland() at filt_timerattach+0x78 user trace: trap %o7=0xc039b344 pc 0xc039b378, sp 0xedebe761 done KDB: enter: panic [ thread pid 1263 tid 100583 ] Stopped at kdb_enter+0x80: ta %xcc, 1 db> Oops. -Kurt From owner-freebsd-hackers@freebsd.org Thu Dec 15 17:12:45 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 63E04C81405 for ; Thu, 15 Dec 2016 17:12:45 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E4DBFEE9 for ; Thu, 15 Dec 2016 17:12:44 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBFHCasH004413 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 15 Dec 2016 19:12:36 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBFHCasH004413 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBFHCabr004412; Thu, 15 Dec 2016 19:12:36 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 15 Dec 2016 19:12:36 +0200 From: Konstantin Belousov To: Kurt Lidl Cc: freebsd-hackers@freebsd.org Subject: Re: How to use sem_timedwait? Message-ID: <20161215171236.GU94325@kib.kiev.ua> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> <1481776532.1889.461.camel@freebsd.org> <44ad49f5-75ec-7c1d-28dc-25df08c67148@pix.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <44ad49f5-75ec-7c1d-28dc-25df08c67148@pix.net> User-Agent: Mutt/1.7.2 (2016-11-26) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 17:12:45 -0000 On Thu, Dec 15, 2016 at 11:47:33AM -0500, Kurt Lidl wrote: > When I run this code on a sparc64, it panics the machine! > > lidl@ton-148: ./x > panic: trap: fast data access mmu miss (kernel) > KDB: stack backtrace: > vpanic() at vpanic+0xfc > panic() at panic+0x20 > trap() at trap+0x554 > -- fast data access mmu miss tar=0xc09eda8f %o7=0xc039b344 -- > userland() at filt_timerattach+0x78 > user trace: trap %o7=0xc039b344 > pc 0xc039b378, sp 0xedebe761 > done > KDB: enter: panic > [ thread pid 1263 tid 100583 ] > Stopped at kdb_enter+0x80: ta %xcc, 1 > db> > > Oops. Could you look up the source line for filt_timerattach+0x78 in your kernel ? From owner-freebsd-hackers@freebsd.org Thu Dec 15 17:15:42 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76CA4C81515 for ; Thu, 15 Dec 2016 17:15:42 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EEC5010EC; Thu, 15 Dec 2016 17:15:41 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBFHFZ3r004897 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 15 Dec 2016 19:15:36 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBFHFZ3r004897 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBFHFYBn004887; Thu, 15 Dec 2016 19:15:34 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 15 Dec 2016 19:15:34 +0200 From: Konstantin Belousov To: Ian Lepore Cc: Goran Meki? , freebsd-hackers@freebsd.org Subject: Re: How to use sem_timedwait? Message-ID: <20161215171534.GV94325@kib.kiev.ua> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> <1481776532.1889.461.camel@freebsd.org> <20161215112050.GO94325@kib.kiev.ua> <1481817343.1889.467.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1481817343.1889.467.camel@freebsd.org> User-Agent: Mutt/1.7.2 (2016-11-26) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 17:15:42 -0000 On Thu, Dec 15, 2016 at 08:55:43AM -0700, Ian Lepore wrote: > It makes me feel a bit better to hear that it's only this machine, > which is pretty old. šI wonder if it has something to do with > overclocking the cpu? šI never have any other kind of timing trouble, > though. š(Given that I work in the field of precision timing, I'm a bit > sensitive to such things.) > > revolution > sysctl kern.timecounter kern.eventtimer > kern.timecounter.tsc_shift: 1 > kern.timecounter.smp_tsc_adjust: 0 > kern.timecounter.smp_tsc: 1 > kern.timecounter.invariant_tsc: 1 > kern.timecounter.fast_gettime: 1 > kern.timecounter.tick: 1 > kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) HPET(950) i8254(0) dummy(-1000000) > kern.timecounter.hardware: TSC-low > kern.timecounter.alloweddeviation: 0 > kern.timecounter.stepwarnings: 0 > kern.timecounter.tc.TSC-low.quality: 1000 > kern.timecounter.tc.TSC-low.frequency: 2125044382 > kern.timecounter.tc.TSC-low.counter: 1611837909 > kern.timecounter.tc.TSC-low.mask: 4294967295 > kern.timecounter.tc.ACPI-fast.quality: 900 > kern.timecounter.tc.ACPI-fast.frequency: 3579545 > kern.timecounter.tc.ACPI-fast.counter: 6548232 > kern.timecounter.tc.ACPI-fast.mask: 16777215 > kern.timecounter.tc.HPET.quality: 950 > kern.timecounter.tc.HPET.frequency: 14318180 > kern.timecounter.tc.HPET.counter: 3623382209 > kern.timecounter.tc.HPET.mask: 4294967295 > kern.timecounter.tc.i8254.quality: 0 > kern.timecounter.tc.i8254.frequency: 1193182 > kern.timecounter.tc.i8254.counter: 18218 > kern.timecounter.tc.i8254.mask: 65535 > kern.eventtimer.periodic: 0 > kern.eventtimer.timer: LAPIC > kern.eventtimer.idletick: 0 > kern.eventtimer.singlemul: 2 > kern.eventtimer.choice: LAPIC(600) HPET(350) HPET1(340) HPET2(340) HPET3(340) i8254(100) RTC(0) > kern.eventtimer.et.HPET3.quality: 340 > kern.eventtimer.et.HPET3.frequency: 14318180 > kern.eventtimer.et.HPET3.flags: 3 > kern.eventtimer.et.HPET2.quality: 340 > kern.eventtimer.et.HPET2.frequency: 14318180 > kern.eventtimer.et.HPET2.flags: 3 > kern.eventtimer.et.HPET1.quality: 340 > kern.eventtimer.et.HPET1.frequency: 14318180 > kern.eventtimer.et.HPET1.flags: 3 > kern.eventtimer.et.HPET.quality: 350 > kern.eventtimer.et.HPET.frequency: 14318180 > kern.eventtimer.et.HPET.flags: 3 > kern.eventtimer.et.RTC.quality: 0 > kern.eventtimer.et.RTC.frequency: 32768 > kern.eventtimer.et.RTC.flags: 17 > kern.eventtimer.et.i8254.quality: 100 > kern.eventtimer.et.i8254.frequency: 1193182 > kern.eventtimer.et.i8254.flags: 1 > kern.eventtimer.et.LAPIC.quality: 600 > kern.eventtimer.et.LAPIC.frequency: 85001804 > kern.eventtimer.et.LAPIC.flags: 7 > > > FreeBSD 10.3-STABLE #5 r308807: Fri Nov 18 09:51:39 MST 2016 > ššššilepore@revolution.hippie.lan:/b/staging/machines/revolution10/obj/b/staging/machines/revolution10/src/sys/REVOLUTION10 amd64 > FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 > VT(vga): text 80x25 > CPU: Intel(R) Xeon(R) CPUšššššššššššW3680šš@ 3.33GHz (4250.09-MHz K8-class CPU) > š Origin="GenuineIntel"ššId=0x206c2ššFamily=0x6ššModel=0x2cššStepping=2 > š Features=0xbfebfbff > š Features2=0x29ee3ff > š AMD Features=0x2c100800 > š AMD Features2=0x1 > š VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID > š TSC: P-state invariant, performance statistics > real memoryšš= 12884901888 (12288 MB) > avail memory = 12425519104 (11849 MB) > Event timer "LAPIC" quality 600 > ACPI APIC Table: <092410 APIC1941> > FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs > FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 SMT threads > It is Westmere machine. If you switch to the HPET hardware for eventtimer, does it feel better ? Also please show output of sysctl dev.cpu.0. You are on stable/10, there were no usage of MWAIT instruction, but still switching to HPET might make a difference. From owner-freebsd-hackers@freebsd.org Thu Dec 15 18:53:50 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B712C820DB for ; Thu, 15 Dec 2016 18:53:50 +0000 (UTC) (envelope-from lidl@pix.net) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254:11::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DCBF1958 for ; Thu, 15 Dec 2016 18:53:50 +0000 (UTC) (envelope-from lidl@pix.net) Received: from torb.pix.net (torb.pix.net [192.168.16.32]) (authenticated bits=0) by hydra.pix.net (8.16.0.19/8.15.2) with ESMTPA id uBFIrnos081281; Thu, 15 Dec 2016 13:53:49 -0500 (EST) (envelope-from lidl@pix.net) Subject: Re: How to use sem_timedwait? To: freebsd-hackers@freebsd.org References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> <1481776532.1889.461.camel@freebsd.org> <44ad49f5-75ec-7c1d-28dc-25df08c67148@pix.net> <20161215171236.GU94325@kib.kiev.ua> From: Kurt Lidl Message-ID: <7a01e4c8-7d58-ad73-4107-4e14b9c89a9d@pix.net> Date: Thu, 15 Dec 2016 13:53:49 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <20161215171236.GU94325@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 18:53:50 -0000 On 12/15/16 12:12 PM, Konstantin Belousov wrote: > On Thu, Dec 15, 2016 at 11:47:33AM -0500, Kurt Lidl wrote: >> When I run this code on a sparc64, it panics the machine! >> >> lidl@ton-148: ./x >> panic: trap: fast data access mmu miss (kernel) >> KDB: stack backtrace: >> vpanic() at vpanic+0xfc >> panic() at panic+0x20 >> trap() at trap+0x554 >> -- fast data access mmu miss tar=0xc09eda8f %o7=0xc039b344 -- >> userland() at filt_timerattach+0x78 >> user trace: trap %o7=0xc039b344 >> pc 0xc039b378, sp 0xedebe761 >> done >> KDB: enter: panic >> [ thread pid 1263 tid 100583 ] >> Stopped at kdb_enter+0x80: ta %xcc, 1 >> db> >> >> Oops. > Could you look up the source line for filt_timerattach+0x78 in your kernel ? I was able to do "call doadump" from ddb, and savecore worked. root@ton-136: kgdb /usr/obj/usr/src/sys/V120/kernel.debug /var/crash/vmcore.1 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "sparc64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: trap: fast data access mmu miss (kernel) KDB: stack backtrace: vpanic() at vpanic+0xfc panic() at panic+0x20 trap() at trap+0x554 -- fast data access mmu miss tar=0xc09eda8f %o7=0xc039b344 -- userland() at filt_timerattach+0x78 user trace: trap %o7=0xc039b344 pc 0xc039b378, sp 0xedebe761 done KDB: enter: panic No symbol "stopped_cpus" in current context. Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/zfs.ko.debug...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. done. Loaded symbols for /boot/kernel/geom_mirror.ko Reading symbols from /boot/kernel/tmpfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/tmpfs.ko.debug...done. done. Loaded symbols for /boot/kernel/tmpfs.ko Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/fdescfs.ko.debug...done. done. Loaded symbols for /boot/kernel/fdescfs.ko #0 0x00000000c03f0040 in doadump (textdump=0) at /usr/src/sys/kern/kern_shutdown.c:297 297 savectx(&dumppcb); (kgdb) bt #0 0x00000000c03f0040 in doadump (textdump=0) at /usr/src/sys/kern/kern_shutdown.c:297 #1 0x00000000c0118f80 in db_fncall (dummy1=1, dummy2=false, dummy3=12289, dummy4=0xedebe410 "??????0\001") at /usr/src/sys/ddb/db_command.c:581 #2 0x00000000c0119484 in db_command (last_cmdp=0xc09ce890, cmd_table=0x0, dopager=1) at /usr/src/sys/ddb/db_command.c:453 #3 0x00000000c0119794 in db_command_loop () at /usr/src/sys/ddb/db_command.c:506 #4 0x00000000c011dfe0 in db_trap (type=, code=0) at /usr/src/sys/ddb/db_main.c:248 #5 0x00000000c043a7c4 in kdb_trap (type=107, code=0, tf=0xedebe920) at /usr/src/sys/kern/subr_kdb.c:654 #6 0x00000000c07a3a28 in trap (tf=0xedebe920) at /usr/src/sys/sparc64/sparc64/trap.c:344 #7 0x00000000c0099060 in tl1_trap () #8 0x00000000c043a4e0 in kdb_enter (why=0x12
, msg=0xc091dfa8 "panic") at /usr/src/sys/kern/subr_kdb.c:442 #9 0x00000000c043a4c8 in kdb_enter (why=0xc091dfa8 "panic", msg=0xc091dfa8 "panic") at /usr/src/sys/kern/subr_kdb.c:441 #10 0x00000000c03f0888 in vpanic (fmt=0xc094cde0 "trap: %s (kernel)", ap=0xedebecf8) at /usr/src/sys/kern/kern_shutdown.c:752 #11 0x00000000c03f0908 in panic (fmt=0xc094cde0 "trap: %s (kernel)") at /usr/src/sys/kern/kern_shutdown.c:690 #12 0x00000000c07a3c9c in trap (tf=0xedebee60) at /usr/src/sys/sparc64/sparc64/trap.c:410 #13 0x00000000c0099060 in tl1_trap () #14 0x00000000c039b378 in filt_timerattach (kn=0x418937) at /usr/src/sys/kern/kern_event.c:675 ---Type to continue, or q to quit--- #15 0x00000000c039b34c in filt_timerattach (kn=0xfffff80016629e80) at /usr/src/sys/kern/kern_event.c:671 #16 0x00000000c039df24 in kqueue_register (kq=0xfffff800010da300, kev=0xedebf230, td=0xfffff800163fa000, waitok=0) at /usr/src/sys/kern/kern_event.c:1316 #17 0x00000000c039e4ec in kqueue_kevent (kq=0xfffff800010da300, td=0xfffff800163fa000, nchanges=1, nevents=1, k_ops=0xedebf5e8, timeout=0x0) at /usr/src/sys/kern/kern_event.c:1001 #18 0x00000000c039f15c in kern_kevent_fp (td=0xfffff800163fa000, fp=0xfffff80001e807d0, nchanges=1, nevents=1, k_ops=0xedebf5e8, timeout=0x0) at /usr/src/sys/kern/kern_event.c:1032 #19 0x00000000c039f21c in kern_kevent (td=0xfffff800163fa000, fd=3, nchanges=1, nevents=1, k_ops=0xedebf5e8, timeout=0x0) at /usr/src/sys/kern/kern_event.c:975 #20 0x00000000c039f3e0 in sys_kevent (td=0xfffff800163fa000, uap=0xedebf768) at /usr/src/sys/kern/kern_event.c:907 #21 0x00000000c07a2a8c in syscall (tf=0xedebf880) at subr_syscall.c:135 #22 0x00000000c0098e40 in tl0_intr () #23 0x0000000000000000 in ?? () (kgdb) up 14 #14 0x00000000c039b378 in filt_timerattach (kn=0x418937) at /usr/src/sys/kern/kern_event.c:675 675 ncallouts = atomic_load_explicit(&kq_ncallouts, memory_order_relaxed); (kgdb) list *0x00000000c039b378 0xc039b378 is in filt_timerattach (/usr/src/sys/kern/kern_event.c:675). 670 671 to = timer2sbintime(kn->kn_sdata, kn->kn_sfflags); 672 if (to < 0) 673 return (EINVAL); 674 675 ncallouts = atomic_load_explicit(&kq_ncallouts, memory_order_relaxed); 676 do { 677 if (ncallouts >= kq_calloutmax) 678 return (ENOMEM); 679 } while (!atomic_compare_exchange_weak_explicit(&kq_ncallouts, This machine is running a stock -head built from a git clone: root@ton-141: uname -a FreeBSD ton.pix.net 12.0-CURRENT FreeBSD 12.0-CURRENT #23 b853a21(master): Sat Dec 3 15:38:11 EST 2016 lidl@ton.pix.net:/usr/obj/usr/src/sys/V120 sparc64 With no changes, except for a custom kernel configuration file. -Kurt From owner-freebsd-hackers@freebsd.org Thu Dec 15 19:17:50 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E3585C82781 for ; Thu, 15 Dec 2016 19:17:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 997BEA7C for ; Thu, 15 Dec 2016 19:17:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBFJHfaT034615 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 15 Dec 2016 21:17:42 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBFJHfaT034615 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBFJHfTo034614; Thu, 15 Dec 2016 21:17:41 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 15 Dec 2016 21:17:41 +0200 From: Konstantin Belousov To: Kurt Lidl Cc: freebsd-hackers@freebsd.org Subject: Re: How to use sem_timedwait? Message-ID: <20161215191741.GX94325@kib.kiev.ua> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> <1481776532.1889.461.camel@freebsd.org> <44ad49f5-75ec-7c1d-28dc-25df08c67148@pix.net> <20161215171236.GU94325@kib.kiev.ua> <7a01e4c8-7d58-ad73-4107-4e14b9c89a9d@pix.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7a01e4c8-7d58-ad73-4107-4e14b9c89a9d@pix.net> User-Agent: Mutt/1.7.2 (2016-11-26) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 19:17:51 -0000 On Thu, Dec 15, 2016 at 01:53:49PM -0500, Kurt Lidl wrote: > On 12/15/16 12:12 PM, Konstantin Belousov wrote: > > On Thu, Dec 15, 2016 at 11:47:33AM -0500, Kurt Lidl wrote: > >> When I run this code on a sparc64, it panics the machine! > >> > >> lidl@ton-148: ./x > >> panic: trap: fast data access mmu miss (kernel) > >> KDB: stack backtrace: > >> vpanic() at vpanic+0xfc > >> panic() at panic+0x20 > >> trap() at trap+0x554 > >> -- fast data access mmu miss tar=0xc09eda8f %o7=0xc039b344 -- > >> userland() at filt_timerattach+0x78 > >> user trace: trap %o7=0xc039b344 > >> pc 0xc039b378, sp 0xedebe761 > >> done > >> KDB: enter: panic > >> [ thread pid 1263 tid 100583 ] > >> Stopped at kdb_enter+0x80: ta %xcc, 1 > >> db> > >> > >> Oops. > > Could you look up the source line for filt_timerattach+0x78 in your kernel ? > > I was able to do "call doadump" from ddb, and savecore worked. > > root@ton-136: kgdb /usr/obj/usr/src/sys/V120/kernel.debug > /var/crash/vmcore.1 > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "sparc64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > panic: trap: fast data access mmu miss (kernel) > KDB: stack backtrace: > vpanic() at vpanic+0xfc > panic() at panic+0x20 > trap() at trap+0x554 > -- fast data access mmu miss tar=0xc09eda8f %o7=0xc039b344 -- > userland() at filt_timerattach+0x78 > user trace: trap %o7=0xc039b344 > pc 0xc039b378, sp 0xedebe761 > done > KDB: enter: panic > > No symbol "stopped_cpus" in current context. > Reading symbols from /boot/kernel/zfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/zfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/zfs.ko > Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from > /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. > done. > Loaded symbols for /boot/kernel/opensolaris.ko > Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from > /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. > done. > Loaded symbols for /boot/kernel/geom_mirror.ko > Reading symbols from /boot/kernel/tmpfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/tmpfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/tmpfs.ko > Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/fdescfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/fdescfs.ko > #0 0x00000000c03f0040 in doadump (textdump=0) at > /usr/src/sys/kern/kern_shutdown.c:297 > 297 savectx(&dumppcb); > (kgdb) bt > #0 0x00000000c03f0040 in doadump (textdump=0) at > /usr/src/sys/kern/kern_shutdown.c:297 > #1 0x00000000c0118f80 in db_fncall (dummy1=1, dummy2=false, dummy3=12289, > dummy4=0xedebe410 "??????0\001") at /usr/src/sys/ddb/db_command.c:581 > #2 0x00000000c0119484 in db_command (last_cmdp=0xc09ce890, > cmd_table=0x0, dopager=1) > at /usr/src/sys/ddb/db_command.c:453 > #3 0x00000000c0119794 in db_command_loop () at > /usr/src/sys/ddb/db_command.c:506 > #4 0x00000000c011dfe0 in db_trap (type=, code=0) > at /usr/src/sys/ddb/db_main.c:248 > #5 0x00000000c043a7c4 in kdb_trap (type=107, code=0, tf=0xedebe920) > at /usr/src/sys/kern/subr_kdb.c:654 > #6 0x00000000c07a3a28 in trap (tf=0xedebe920) at > /usr/src/sys/sparc64/sparc64/trap.c:344 > #7 0x00000000c0099060 in tl1_trap () > #8 0x00000000c043a4e0 in kdb_enter (why=0x12
, > msg=0xc091dfa8 "panic") at /usr/src/sys/kern/subr_kdb.c:442 > #9 0x00000000c043a4c8 in kdb_enter (why=0xc091dfa8 "panic", > msg=0xc091dfa8 "panic") > at /usr/src/sys/kern/subr_kdb.c:441 > #10 0x00000000c03f0888 in vpanic (fmt=0xc094cde0 "trap: %s (kernel)", > ap=0xedebecf8) > at /usr/src/sys/kern/kern_shutdown.c:752 > #11 0x00000000c03f0908 in panic (fmt=0xc094cde0 "trap: %s (kernel)") > at /usr/src/sys/kern/kern_shutdown.c:690 > #12 0x00000000c07a3c9c in trap (tf=0xedebee60) at > /usr/src/sys/sparc64/sparc64/trap.c:410 > #13 0x00000000c0099060 in tl1_trap () > #14 0x00000000c039b378 in filt_timerattach (kn=0x418937) at > /usr/src/sys/kern/kern_event.c:675 > ---Type to continue, or q to quit--- > #15 0x00000000c039b34c in filt_timerattach (kn=0xfffff80016629e80) > at /usr/src/sys/kern/kern_event.c:671 > #16 0x00000000c039df24 in kqueue_register (kq=0xfffff800010da300, > kev=0xedebf230, > td=0xfffff800163fa000, waitok=0) at /usr/src/sys/kern/kern_event.c:1316 > #17 0x00000000c039e4ec in kqueue_kevent (kq=0xfffff800010da300, > td=0xfffff800163fa000, > nchanges=1, nevents=1, k_ops=0xedebf5e8, timeout=0x0) at > /usr/src/sys/kern/kern_event.c:1001 > #18 0x00000000c039f15c in kern_kevent_fp (td=0xfffff800163fa000, > fp=0xfffff80001e807d0, > nchanges=1, nevents=1, k_ops=0xedebf5e8, timeout=0x0) at > /usr/src/sys/kern/kern_event.c:1032 > #19 0x00000000c039f21c in kern_kevent (td=0xfffff800163fa000, fd=3, > nchanges=1, nevents=1, > k_ops=0xedebf5e8, timeout=0x0) at /usr/src/sys/kern/kern_event.c:975 > #20 0x00000000c039f3e0 in sys_kevent (td=0xfffff800163fa000, uap=0xedebf768) > at /usr/src/sys/kern/kern_event.c:907 > #21 0x00000000c07a2a8c in syscall (tf=0xedebf880) at subr_syscall.c:135 > #22 0x00000000c0098e40 in tl0_intr () > #23 0x0000000000000000 in ?? () > (kgdb) up 14 > #14 0x00000000c039b378 in filt_timerattach (kn=0x418937) at > /usr/src/sys/kern/kern_event.c:675 > 675 ncallouts = atomic_load_explicit(&kq_ncallouts, memory_order_relaxed); > (kgdb) list *0x00000000c039b378 > 0xc039b378 is in filt_timerattach (/usr/src/sys/kern/kern_event.c:675). > 670 > 671 to = timer2sbintime(kn->kn_sdata, kn->kn_sfflags); > 672 if (to < 0) > 673 return (EINVAL); > 674 > 675 ncallouts = atomic_load_explicit(&kq_ncallouts, memory_order_relaxed); > 676 do { > 677 if (ncallouts >= kq_calloutmax) > 678 return (ENOMEM); > 679 } while (!atomic_compare_exchange_weak_explicit(&kq_ncallouts, Note that kgdb has some troubles interpreting the stack, since it reported two frames for filt_timerattach(), frames 14 and 15. I would believe into the ddb trace more. As such, could you compare filt_timerattach+0x78 address with the pc for frames 14, 15 to see which one more trustwory. That said, I will be not too surprised if the issue is indeed in the emulation of C11 atomics on old gcc. The reasons to use stdatomic there was silly, so if this breaks sparc64 I would just switch to atomic.h. Could you try the patch below and see if the problem goes away ? diff --git a/sys/kern/kern_event.c b/sys/kern/kern_event.c index 36fe20fbc91..b734cf271bc 100644 --- a/sys/kern/kern_event.c +++ b/sys/kern/kern_event.c @@ -48,7 +48,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include #include @@ -69,6 +68,7 @@ __FBSDID("$FreeBSD$"); #ifdef KTRACE #include #endif +#include #include @@ -188,7 +188,7 @@ static struct filterops user_filtops = { }; static uma_zone_t knote_zone; -static atomic_uint kq_ncallouts = ATOMIC_VAR_INIT(0); +static unsigned int kq_ncallouts = 0; static unsigned int kq_calloutmax = 4 * 1024; SYSCTL_UINT(_kern, OID_AUTO, kq_calloutmax, CTLFLAG_RW, &kq_calloutmax, 0, "Maximum number of callouts allocated for kqueue"); @@ -672,13 +672,11 @@ filt_timerattach(struct knote *kn) if (to < 0) return (EINVAL); - ncallouts = atomic_load_explicit(&kq_ncallouts, memory_order_relaxed); do { + ncallouts = kq_ncallouts; if (ncallouts >= kq_calloutmax) return (ENOMEM); - } while (!atomic_compare_exchange_weak_explicit(&kq_ncallouts, - &ncallouts, ncallouts + 1, memory_order_relaxed, - memory_order_relaxed)); + } while (!atomic_cmpset_int(&kq_ncallouts, ncallouts, ncallouts + 1)); kn->kn_flags |= EV_CLEAR; /* automatically set */ kn->kn_status &= ~KN_DETACHED; /* knlist_add clears it */ @@ -703,7 +701,7 @@ filt_timerdetach(struct knote *kn) callout_drain(calloutp); free(calloutp, M_KQUEUE); free(kn->kn_ptr.p_nexttime, M_KQUEUE); - old = atomic_fetch_sub_explicit(&kq_ncallouts, 1, memory_order_relaxed); + old = atomic_fetchadd_int(&kq_ncallouts, -1); KASSERT(old > 0, ("Number of callouts cannot become negative")); kn->kn_status |= KN_DETACHED; /* knlist_remove sets it */ } From owner-freebsd-hackers@freebsd.org Fri Dec 16 03:48:13 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EBE81C82AAB for ; Fri, 16 Dec 2016 03:48:13 +0000 (UTC) (envelope-from lidl@pix.net) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254:11::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C5BC119F2 for ; Fri, 16 Dec 2016 03:48:13 +0000 (UTC) (envelope-from lidl@pix.net) Received: from torb.pix.net (torb.pix.net [192.168.16.32]) (authenticated bits=0) by hydra.pix.net (8.16.0.19/8.15.2) with ESMTPA id uBG3mCSP001924; Thu, 15 Dec 2016 22:48:12 -0500 (EST) (envelope-from lidl@pix.net) Subject: Re: How to use sem_timedwait? To: freebsd-hackers@freebsd.org References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> <1481776532.1889.461.camel@freebsd.org> <44ad49f5-75ec-7c1d-28dc-25df08c67148@pix.net> <20161215171236.GU94325@kib.kiev.ua> <7a01e4c8-7d58-ad73-4107-4e14b9c89a9d@pix.net> <20161215191741.GX94325@kib.kiev.ua> From: Kurt Lidl Message-ID: <367ab8d2-06fc-92d0-97dc-09b7f8cf2618@pix.net> Date: Thu, 15 Dec 2016 22:48:12 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <20161215191741.GX94325@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2016 03:48:14 -0000 On 12/15/16 2:17 PM, Konstantin Belousov wrote: > On Thu, Dec 15, 2016 at 01:53:49PM -0500, Kurt Lidl wrote: >> On 12/15/16 12:12 PM, Konstantin Belousov wrote: >>> On Thu, Dec 15, 2016 at 11:47:33AM -0500, Kurt Lidl wrote: >>>> When I run this code on a sparc64, it panics the machine! >>>> >>>> lidl@ton-148: ./x >>>> panic: trap: fast data access mmu miss (kernel) >>>> KDB: stack backtrace: >>>> vpanic() at vpanic+0xfc >>>> panic() at panic+0x20 >>>> trap() at trap+0x554 >>>> -- fast data access mmu miss tar=0xc09eda8f %o7=0xc039b344 -- >>>> userland() at filt_timerattach+0x78 >>>> user trace: trap %o7=0xc039b344 >>>> pc 0xc039b378, sp 0xedebe761 >>>> done >>>> KDB: enter: panic >>>> [ thread pid 1263 tid 100583 ] >>>> Stopped at kdb_enter+0x80: ta %xcc, 1 >>>> db> >>>> >>>> Oops. >>> Could you look up the source line for filt_timerattach+0x78 in your kernel ? >> >> I was able to do "call doadump" from ddb, and savecore worked. >> >> root@ton-136: kgdb /usr/obj/usr/src/sys/V120/kernel.debug >> /var/crash/vmcore.1 >> >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and you are >> welcome to change it and/or distribute copies of it under certain >> conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for details. >> This GDB was configured as "sparc64-marcel-freebsd"... >> >> Unread portion of the kernel message buffer: >> panic: trap: fast data access mmu miss (kernel) >> KDB: stack backtrace: >> vpanic() at vpanic+0xfc >> panic() at panic+0x20 >> trap() at trap+0x554 >> -- fast data access mmu miss tar=0xc09eda8f %o7=0xc039b344 -- >> userland() at filt_timerattach+0x78 >> user trace: trap %o7=0xc039b344 >> pc 0xc039b378, sp 0xedebe761 >> done >> KDB: enter: panic >> >> No symbol "stopped_cpus" in current context. >> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/zfs.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/zfs.ko >> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/opensolaris.ko >> Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/geom_mirror.ko >> Reading symbols from /boot/kernel/tmpfs.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/tmpfs.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/tmpfs.ko >> Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/fdescfs.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/fdescfs.ko >> #0 0x00000000c03f0040 in doadump (textdump=0) at >> /usr/src/sys/kern/kern_shutdown.c:297 >> 297 savectx(&dumppcb); >> (kgdb) bt >> #0 0x00000000c03f0040 in doadump (textdump=0) at >> /usr/src/sys/kern/kern_shutdown.c:297 >> #1 0x00000000c0118f80 in db_fncall (dummy1=1, dummy2=false, dummy3=12289, >> dummy4=0xedebe410 "??????0\001") at /usr/src/sys/ddb/db_command.c:581 >> #2 0x00000000c0119484 in db_command (last_cmdp=0xc09ce890, >> cmd_table=0x0, dopager=1) >> at /usr/src/sys/ddb/db_command.c:453 >> #3 0x00000000c0119794 in db_command_loop () at >> /usr/src/sys/ddb/db_command.c:506 >> #4 0x00000000c011dfe0 in db_trap (type=, code=0) >> at /usr/src/sys/ddb/db_main.c:248 >> #5 0x00000000c043a7c4 in kdb_trap (type=107, code=0, tf=0xedebe920) >> at /usr/src/sys/kern/subr_kdb.c:654 >> #6 0x00000000c07a3a28 in trap (tf=0xedebe920) at >> /usr/src/sys/sparc64/sparc64/trap.c:344 >> #7 0x00000000c0099060 in tl1_trap () >> #8 0x00000000c043a4e0 in kdb_enter (why=0x12
, >> msg=0xc091dfa8 "panic") at /usr/src/sys/kern/subr_kdb.c:442 >> #9 0x00000000c043a4c8 in kdb_enter (why=0xc091dfa8 "panic", >> msg=0xc091dfa8 "panic") >> at /usr/src/sys/kern/subr_kdb.c:441 >> #10 0x00000000c03f0888 in vpanic (fmt=0xc094cde0 "trap: %s (kernel)", >> ap=0xedebecf8) >> at /usr/src/sys/kern/kern_shutdown.c:752 >> #11 0x00000000c03f0908 in panic (fmt=0xc094cde0 "trap: %s (kernel)") >> at /usr/src/sys/kern/kern_shutdown.c:690 >> #12 0x00000000c07a3c9c in trap (tf=0xedebee60) at >> /usr/src/sys/sparc64/sparc64/trap.c:410 >> #13 0x00000000c0099060 in tl1_trap () >> #14 0x00000000c039b378 in filt_timerattach (kn=0x418937) at >> /usr/src/sys/kern/kern_event.c:675 >> ---Type to continue, or q to quit--- >> #15 0x00000000c039b34c in filt_timerattach (kn=0xfffff80016629e80) >> at /usr/src/sys/kern/kern_event.c:671 >> #16 0x00000000c039df24 in kqueue_register (kq=0xfffff800010da300, >> kev=0xedebf230, >> td=0xfffff800163fa000, waitok=0) at /usr/src/sys/kern/kern_event.c:1316 >> #17 0x00000000c039e4ec in kqueue_kevent (kq=0xfffff800010da300, >> td=0xfffff800163fa000, >> nchanges=1, nevents=1, k_ops=0xedebf5e8, timeout=0x0) at >> /usr/src/sys/kern/kern_event.c:1001 >> #18 0x00000000c039f15c in kern_kevent_fp (td=0xfffff800163fa000, >> fp=0xfffff80001e807d0, >> nchanges=1, nevents=1, k_ops=0xedebf5e8, timeout=0x0) at >> /usr/src/sys/kern/kern_event.c:1032 >> #19 0x00000000c039f21c in kern_kevent (td=0xfffff800163fa000, fd=3, >> nchanges=1, nevents=1, >> k_ops=0xedebf5e8, timeout=0x0) at /usr/src/sys/kern/kern_event.c:975 >> #20 0x00000000c039f3e0 in sys_kevent (td=0xfffff800163fa000, uap=0xedebf768) >> at /usr/src/sys/kern/kern_event.c:907 >> #21 0x00000000c07a2a8c in syscall (tf=0xedebf880) at subr_syscall.c:135 >> #22 0x00000000c0098e40 in tl0_intr () >> #23 0x0000000000000000 in ?? () >> (kgdb) up 14 >> #14 0x00000000c039b378 in filt_timerattach (kn=0x418937) at >> /usr/src/sys/kern/kern_event.c:675 >> 675 ncallouts = atomic_load_explicit(&kq_ncallouts, memory_order_relaxed); >> (kgdb) list *0x00000000c039b378 >> 0xc039b378 is in filt_timerattach (/usr/src/sys/kern/kern_event.c:675). >> 670 >> 671 to = timer2sbintime(kn->kn_sdata, kn->kn_sfflags); >> 672 if (to < 0) >> 673 return (EINVAL); >> 674 >> 675 ncallouts = atomic_load_explicit(&kq_ncallouts, memory_order_relaxed); >> 676 do { >> 677 if (ncallouts >= kq_calloutmax) >> 678 return (ENOMEM); >> 679 } while (!atomic_compare_exchange_weak_explicit(&kq_ncallouts, > Note that kgdb has some troubles interpreting the stack, since it reported > two frames for filt_timerattach(), frames 14 and 15. I would believe into > the ddb trace more. > > As such, could you compare filt_timerattach+0x78 address with the pc for > frames 14, 15 to see which one more trustwory. > > That said, I will be not too surprised if the issue is indeed in the > emulation of C11 atomics on old gcc. The reasons to use stdatomic there > was silly, so if this breaks sparc64 I would just switch to atomic.h. > > Could you try the patch below and see if the problem goes away ? With that patch applied (and no other changes to the kernel), it no longer panics: lidl@ton-156: ./x nsec = 10078193981 for 10000 loops of 1000000 nsec (Doesn't keep great time, but it does work.) Thanks! -Kurt From owner-freebsd-hackers@freebsd.org Fri Dec 16 18:07:02 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F649C83197 for ; Fri, 16 Dec 2016 18:07:02 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 02C5E1CCF for ; Fri, 16 Dec 2016 18:07:01 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: 6fc45d57-c3ba-11e6-8c89-112185c90658 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id 6fc45d57-c3ba-11e6-8c89-112185c90658; Fri, 16 Dec 2016 18:06:59 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id uBGI6rYw003943; Fri, 16 Dec 2016 11:06:53 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1481911613.1972.74.camel@freebsd.org> Subject: Re: How to use sem_timedwait? From: Ian Lepore To: Konstantin Belousov Cc: freebsd-hackers@freebsd.org Date: Fri, 16 Dec 2016 11:06:53 -0700 In-Reply-To: <20161215171534.GV94325@kib.kiev.ua> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> <1481776532.1889.461.camel@freebsd.org> <20161215112050.GO94325@kib.kiev.ua> <1481817343.1889.467.camel@freebsd.org> <20161215171534.GV94325@kib.kiev.ua> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2016 18:07:02 -0000 On Thu, 2016-12-15 at 19:15 +0200, Konstantin Belousov wrote: > On Thu, Dec 15, 2016 at 08:55:43AM -0700, Ian Lepore wrote: > > > > It makes me feel a bit better to hear that it's only this machine, > > which is pretty old.  I wonder if it has something to do with > > overclocking the cpu?  I never have any other kind of timing > > trouble, > > though.  (Given that I work in the field of precision timing, I'm a > > bit > > sensitive to such things.) > > > > revolution > sysctl kern.timecounter kern.eventtimer > > kern.timecounter.tsc_shift: 1 > > kern.timecounter.smp_tsc_adjust: 0 > > kern.timecounter.smp_tsc: 1 > > kern.timecounter.invariant_tsc: 1 > > kern.timecounter.fast_gettime: 1 > > kern.timecounter.tick: 1 > > kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) HPET(950) > > i8254(0) dummy(-1000000) > > kern.timecounter.hardware: TSC-low > > kern.timecounter.alloweddeviation: 0 > > kern.timecounter.stepwarnings: 0 > > kern.timecounter.tc.TSC-low.quality: 1000 > > kern.timecounter.tc.TSC-low.frequency: 2125044382 > > kern.timecounter.tc.TSC-low.counter: 1611837909 > > kern.timecounter.tc.TSC-low.mask: 4294967295 > > kern.timecounter.tc.ACPI-fast.quality: 900 > > kern.timecounter.tc.ACPI-fast.frequency: 3579545 > > kern.timecounter.tc.ACPI-fast.counter: 6548232 > > kern.timecounter.tc.ACPI-fast.mask: 16777215 > > kern.timecounter.tc.HPET.quality: 950 > > kern.timecounter.tc.HPET.frequency: 14318180 > > kern.timecounter.tc.HPET.counter: 3623382209 > > kern.timecounter.tc.HPET.mask: 4294967295 > > kern.timecounter.tc.i8254.quality: 0 > > kern.timecounter.tc.i8254.frequency: 1193182 > > kern.timecounter.tc.i8254.counter: 18218 > > kern.timecounter.tc.i8254.mask: 65535 > > kern.eventtimer.periodic: 0 > > kern.eventtimer.timer: LAPIC > > kern.eventtimer.idletick: 0 > > kern.eventtimer.singlemul: 2 > > kern.eventtimer.choice: LAPIC(600) HPET(350) HPET1(340) HPET2(340) > > HPET3(340) i8254(100) RTC(0) > > kern.eventtimer.et.HPET3.quality: 340 > > kern.eventtimer.et.HPET3.frequency: 14318180 > > kern.eventtimer.et.HPET3.flags: 3 > > kern.eventtimer.et.HPET2.quality: 340 > > kern.eventtimer.et.HPET2.frequency: 14318180 > > kern.eventtimer.et.HPET2.flags: 3 > > kern.eventtimer.et.HPET1.quality: 340 > > kern.eventtimer.et.HPET1.frequency: 14318180 > > kern.eventtimer.et.HPET1.flags: 3 > > kern.eventtimer.et.HPET.quality: 350 > > kern.eventtimer.et.HPET.frequency: 14318180 > > kern.eventtimer.et.HPET.flags: 3 > > kern.eventtimer.et.RTC.quality: 0 > > kern.eventtimer.et.RTC.frequency: 32768 > > kern.eventtimer.et.RTC.flags: 17 > > kern.eventtimer.et.i8254.quality: 100 > > kern.eventtimer.et.i8254.frequency: 1193182 > > kern.eventtimer.et.i8254.flags: 1 > > kern.eventtimer.et.LAPIC.quality: 600 > > kern.eventtimer.et.LAPIC.frequency: 85001804 > > kern.eventtimer.et.LAPIC.flags: 7 > > > > > > FreeBSD 10.3-STABLE #5 r308807: Fri Nov 18 09:51:39 MST 2016 > >     ilepore@revolution.hippie.lan:/b/staging/machines/revolution10/ > > obj/b/staging/machines/revolution10/src/sys/REVOLUTION10 amd64 > > FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) > > 20140512 > > VT(vga): text 80x25 > > CPU: Intel(R) Xeon(R) CPU           W3680  @ 3.33GHz (4250.09-MHz > > K8-class CPU) > >   > > Origin="GenuineIntel"  Id=0x206c2  Family=0x6  Model=0x2c  Stepping > > =2 > >   > > Features=0xbfebfbff > R,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT, > > TM,PBE> > >   > > Features2=0x29ee3ff > 2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AESNI> > >   AMD Features=0x2c100800 > >   AMD Features2=0x1 > >   VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID > >   TSC: P-state invariant, performance statistics > > real memory  = 12884901888 (12288 MB) > > avail memory = 12425519104 (11849 MB) > > Event timer "LAPIC" quality 600 > > ACPI APIC Table: <092410 APIC1941> > > FreeBSD/SMP: Multiprocessor System Detected: 12 CPUs > > FreeBSD/SMP: 1 package(s) x 6 core(s) x 2 SMT threads > > > It is Westmere machine.  If you switch to the HPET hardware for > eventtimer, > does it feel better ? > > Also please show output of sysctl dev.cpu.0.  You are on stable/10, > there > were no usage of MWAIT instruction, but still switching to HPET might > make > a difference. > _ I tried LAPIC, HPET, and i8254 for eventtimer.hardware, there was no change with any of them.  I tried turning periodic on, still no change.   Changing alloweddeviation made no change.  I tried different hardware choices for timecounter too (not that timecounting is showing any signs of trouble on this machine), no change. revolution > sysctl dev.cpu.0 dev.cpu.0.cx_usage: 100.00% 0.00% 0.00% last 303us dev.cpu.0.cx_lowest: C1 dev.cpu.0.cx_supported: C1/1/32 C2/2/96 C3/3/128 dev.cpu.0.freq_levels: 3334/143000 3333/130000 3200/117000 3067/105000 2933/94000 2800/85000 2667/76000 2533/68000 2400/61000 2267/54000 2133/48000 2000/43000 1867/39000 1733/35000 1600/32000 dev.cpu.0.freq: 3333 dev.cpu.0.temperature: 69.0C dev.cpu.0.coretemp.throttle_log: 0 dev.cpu.0.coretemp.tjmax: 101.0C dev.cpu.0.coretemp.resolution: 1 dev.cpu.0.coretemp.delta: 32 dev.cpu.0.%parent: acpi0 dev.cpu.0.%pnpinfo: _HID=none _UID=0 dev.cpu.0.%location: handle=\_PR_.P001 dev.cpu.0.%driver: cpu dev.cpu.0.%desc: ACPI CPU I also ran the test program on a couple servers at $work runing 10.3- stable, and they both showed identical behavior, with the result in the same range as on this machine: nsec = 9315538211 for 1000000 loops of 10000 nsec. One of those machines is running as a vmware guest: FreeBSD 10.3-STABLE-20161014 #2 r307350M: Mon Oct 17 18:07:23 MDT 2016 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 CPU: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (2197.45-MHz K8-class CPU)   Origin="GenuineIntel"  Id=0x206d2  Family=0x6  Model=0x2d  Stepping=2   Features=0xfa3fbff   Features2=0x9fba2203   AMD Features=0x28100800   AMD Features2=0x1   Structured Extended Features=0x2   TSC: P-state invariant Hypervisor: Origin = "VMwareVMware" real memory  = 2147483648 (2048 MB) avail memory = 2050031616 (1955 MB) Event timer "LAPIC" quality 600 and the other is running on bare metal: FreeBSD 10.3-STABLE-20161014 #2 r307350M: Mon Oct 17 18:30:37 MDT 2016 CPU: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz (2660.05-MHz K8-class CPU)   Origin="GenuineIntel"  Id=0x106a5  Family=0x6  Model=0x1a  Stepping=5   Features=0xbfebfbff   Features2=0x9ce3bd   AMD Features=0x28100800   AMD Features2=0x1   VT-x: PAT,HLT,MTF,PAUSE,EPT,VPID   TSC: P-state invariant, performance statistics real memory  = 206158430208 (196608 MB) avail memory = 200380227584 (191097 MB) Event timer "LAPIC" quality 400 -- Ian From owner-freebsd-hackers@freebsd.org Fri Dec 16 18:46:00 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 811F6C83E83 for ; Fri, 16 Dec 2016 18:46:00 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0624B1EC; Fri, 16 Dec 2016 18:45:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBGIjrIa031765 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 16 Dec 2016 20:45:53 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBGIjrIa031765 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBGIjrY4031764; Fri, 16 Dec 2016 20:45:53 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 16 Dec 2016 20:45:53 +0200 From: Konstantin Belousov To: Ian Lepore Cc: freebsd-hackers@freebsd.org Subject: Re: How to use sem_timedwait? Message-ID: <20161216184553.GC94325@kib.kiev.ua> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> <1481776532.1889.461.camel@freebsd.org> <20161215112050.GO94325@kib.kiev.ua> <1481817343.1889.467.camel@freebsd.org> <20161215171534.GV94325@kib.kiev.ua> <1481911613.1972.74.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1481911613.1972.74.camel@freebsd.org> User-Agent: Mutt/1.7.2 (2016-11-26) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2016 18:46:00 -0000 On Fri, Dec 16, 2016 at 11:06:53AM -0700, Ian Lepore wrote: > I tried LAPIC, HPET, and i8254 for eventtimer.hardware, there was no > change with any of them. šI tried turning periodic on, still no change. > š Changing alloweddeviation made no change. šI tried different hardware > choices for timecounter too (not that timecounting is showing any signs > of trouble on this machine), no change. > > revolution > sysctl dev.cpu.0 > dev.cpu.0.cx_usage: 100.00% 0.00% 0.00% last 303us > dev.cpu.0.cx_lowest: C1 > dev.cpu.0.cx_supported: C1/1/32 C2/2/96 C3/3/128 > dev.cpu.0.freq_levels: 3334/143000 3333/130000 3200/117000 3067/105000 > 2933/94000 2800/85000 2667/76000 2533/68000 2400/61000 2267/54000 > 2133/48000 2000/43000 1867/39000 1733/35000 1600/32000 > dev.cpu.0.freq: 3333 > dev.cpu.0.temperature: 69.0C > dev.cpu.0.coretemp.throttle_log: 0 > dev.cpu.0.coretemp.tjmax: 101.0C > dev.cpu.0.coretemp.resolution: 1 > dev.cpu.0.coretemp.delta: 32 > dev.cpu.0.%parent: acpi0 > dev.cpu.0.%pnpinfo: _HID=none _UID=0 > dev.cpu.0.%location: handle=\_PR_.P001 > dev.cpu.0.%driver: cpu > dev.cpu.0.%desc: ACPI CPU > > I also ran the test program on a couple servers at $work runing 10.3- > stable, and they both showed identical behavior, with the result in the > same range as on this machine: nsec =š9315538211 for 1000000 loops of > 10000 nsec. > > One of those machines is running as a vmware guest: > > FreeBSD 10.3-STABLE-20161014 #2 r307350M: Mon Oct 17 18:07:23 MDT 2016 > FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 > CPU: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (2197.45-MHz K8-class CPU) > š Origin="GenuineIntel"ššId=0x206d2ššFamily=0x6ššModel=0x2dššStepping=2 > š Features=0xfa3fbff > š Features2=0x9fba2203 > š AMD Features=0x28100800 > š AMD Features2=0x1 > š Structured Extended Features=0x2 > š TSC: P-state invariant > Hypervisor: Origin = "VMwareVMware" > real memoryšš= 2147483648 (2048 MB) > avail memory = 2050031616 (1955 MB) > Event timer "LAPIC" quality 600 > > and the other is running on bare metal: > > FreeBSD 10.3-STABLE-20161014 #2 r307350M: Mon Oct 17 18:30:37 MDT 2016 > CPU: Intel(R) Xeon(R) CPUšššššššššššX5550šš@ 2.67GHz (2660.05-MHz K8-class CPU) > š Origin="GenuineIntel"ššId=0x106a5ššFamily=0x6ššModel=0x1aššStepping=5 > š Features=0xbfebfbff > š Features2=0x9ce3bd > š AMD Features=0x28100800 > š AMD Features2=0x1 > š VT-x: PAT,HLT,MTF,PAUSE,EPT,VPID > š TSC: P-state invariant, performance statistics > real memoryšš= 206158430208 (196608 MB) > avail memory = 200380227584 (191097 MB) > Event timer "LAPIC" quality 400 Am I right that you only tested on stable/10 machines ? What you see and I do not, might be due to the missed MFC of r296775, which commit message claims that it fixes truncation in converions. Below is the commit merged to stable/10, could you test it ? I do not have any stable/10 machine. Index: sys/kern/kern_event.c =================================================================== --- sys/kern/kern_event.c (revision 310169) +++ sys/kern/kern_event.c (working copy) @@ -556,34 +556,59 @@ knote_fork(struct knlist *list, int pid) #define NOTE_TIMER_PRECMASK (NOTE_SECONDS|NOTE_MSECONDS|NOTE_USECONDS| \ NOTE_NSECONDS) -static __inline sbintime_t +static sbintime_t timer2sbintime(intptr_t data, int flags) { - sbintime_t modifier; + /* + * Macros for converting to the fractional second portion of an + * sbintime_t using 64bit multiplication to improve precision. + */ +#define NS_TO_SBT(ns) (((ns) * (((uint64_t)1 << 63) / 500000000)) >> 32) +#define US_TO_SBT(us) (((us) * (((uint64_t)1 << 63) / 500000)) >> 32) +#define MS_TO_SBT(ms) (((ms) * (((uint64_t)1 << 63) / 500)) >> 32) switch (flags & NOTE_TIMER_PRECMASK) { case NOTE_SECONDS: - modifier = SBT_1S; - break; +#ifdef __LP64__ + if (data > (SBT_MAX / SBT_1S)) + return SBT_MAX; +#endif + return ((sbintime_t)data << 32); case NOTE_MSECONDS: /* FALLTHROUGH */ case 0: - modifier = SBT_1MS; - break; + if (data >= 1000) { + int64_t secs = data / 1000; +#ifdef __LP64__ + if (secs > (SBT_MAX / SBT_1S)) + return SBT_MAX; +#endif + return (secs << 32 | MS_TO_SBT(data % 1000)); + } + return MS_TO_SBT(data); case NOTE_USECONDS: - modifier = SBT_1US; - break; + if (data >= 1000000) { + int64_t secs = data / 1000000; +#ifdef __LP64__ + if (secs > (SBT_MAX / SBT_1S)) + return SBT_MAX; +#endif + return (secs << 32 | US_TO_SBT(data % 1000000)); + } + return US_TO_SBT(data); case NOTE_NSECONDS: - modifier = SBT_1NS; + if (data >= 1000000000) { + int64_t secs = data / 1000000000; +#ifdef __LP64__ + if (secs > (SBT_MAX / SBT_1S)) + return SBT_MAX; +#endif + return (secs << 32 | US_TO_SBT(data % 1000000000)); + } + return NS_TO_SBT(data); + default: break; - default: - return (-1); } - -#ifdef __LP64__ - if (data > SBT_MAX / modifier) - return (SBT_MAX); -#endif - return (modifier * data); + return (-1); } static void Index: . =================================================================== --- . (revision 310169) +++ . (working copy) Property changes on: . ___________________________________________________________________ Modified: svn:mergeinfo ## -0,0 +0,1 ## Merged /head:r296775 From owner-freebsd-hackers@freebsd.org Fri Dec 16 23:31:01 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3843C834D4 for ; Fri, 16 Dec 2016 23:31:01 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1a.eu.mailhop.org (outbound1a.eu.mailhop.org [52.58.109.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 603F3126 for ; Fri, 16 Dec 2016 23:31:01 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: af870281-c3e7-11e6-9357-bffcd86bd944 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.eu.mailhop.org (Halon) with ESMTPSA id af870281-c3e7-11e6-9357-bffcd86bd944; Fri, 16 Dec 2016 23:30:54 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id uBGNUm5S004514; Fri, 16 Dec 2016 16:30:48 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1481931048.1343.2.camel@freebsd.org> Subject: Re: How to use sem_timedwait? From: Ian Lepore To: Konstantin Belousov Cc: freebsd-hackers@freebsd.org Date: Fri, 16 Dec 2016 16:30:48 -0700 In-Reply-To: <20161216184553.GC94325@kib.kiev.ua> References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> <1481776532.1889.461.camel@freebsd.org> <20161215112050.GO94325@kib.kiev.ua> <1481817343.1889.467.camel@freebsd.org> <20161215171534.GV94325@kib.kiev.ua> <1481911613.1972.74.camel@freebsd.org> <20161216184553.GC94325@kib.kiev.ua> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2016 23:31:01 -0000 On Fri, 2016-12-16 at 20:45 +0200, Konstantin Belousov wrote: > On Fri, Dec 16, 2016 at 11:06:53AM -0700, Ian Lepore wrote: > > > > I tried LAPIC, HPET, and i8254 for eventtimer.hardware, there was > > no > > change with any of them.  I tried turning periodic on, still no > > change. > >   Changing alloweddeviation made no change.  I tried different > > hardware > > choices for timecounter too (not that timecounting is showing any > > signs > > of trouble on this machine), no change. > > > > revolution > sysctl dev.cpu.0 > > dev.cpu.0.cx_usage: 100.00% 0.00% 0.00% last 303us > > dev.cpu.0.cx_lowest: C1 > > dev.cpu.0.cx_supported: C1/1/32 C2/2/96 C3/3/128 > > dev.cpu.0.freq_levels: 3334/143000 3333/130000 3200/117000 > > 3067/105000 > > 2933/94000 2800/85000 2667/76000 2533/68000 2400/61000 2267/54000 > > 2133/48000 2000/43000 1867/39000 1733/35000 1600/32000 > > dev.cpu.0.freq: 3333 > > dev.cpu.0.temperature: 69.0C > > dev.cpu.0.coretemp.throttle_log: 0 > > dev.cpu.0.coretemp.tjmax: 101.0C > > dev.cpu.0.coretemp.resolution: 1 > > dev.cpu.0.coretemp.delta: 32 > > dev.cpu.0.%parent: acpi0 > > dev.cpu.0.%pnpinfo: _HID=none _UID=0 > > dev.cpu.0.%location: handle=\_PR_.P001 > > dev.cpu.0.%driver: cpu > > dev.cpu.0.%desc: ACPI CPU > > > > I also ran the test program on a couple servers at $work runing > > 10.3- > > stable, and they both showed identical behavior, with the result in > > the > > same range as on this machine: nsec = 9315538211 for 1000000 loops > > of > > 10000 nsec. > > > > One of those machines is running as a vmware guest: > > > > FreeBSD 10.3-STABLE-20161014 #2 r307350M: Mon Oct 17 18:07:23 MDT > > 2016 > > FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) > > 20140512 > > CPU: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (2197.45-MHz K8- > > class CPU) > >   > > Origin="GenuineIntel"  Id=0x206d2  Family=0x6  Model=0x2d  Stepping > > =2 > >   > > Features=0xfa3fbff > ,PGE,MCA,CMOV,PAT,PSE36,DTS,MMX,FXSR,SSE,SSE2,SS> > >   > > Features2=0x9fba2203 > 2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,HV> > >   AMD Features=0x28100800 > >   AMD Features2=0x1 > >   Structured Extended Features=0x2 > >   TSC: P-state invariant > > Hypervisor: Origin = "VMwareVMware" > > real memory  = 2147483648 (2048 MB) > > avail memory = 2050031616 (1955 MB) > > Event timer "LAPIC" quality 600 > > > > and the other is running on bare metal: > > > > FreeBSD 10.3-STABLE-20161014 #2 r307350M: Mon Oct 17 18:30:37 MDT > > 2016 > > CPU: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz (2660.05-MHz > > K8-class CPU) > >   > > Origin="GenuineIntel"  Id=0x106a5  Family=0x6  Model=0x1a  Stepping > > =5 > >   > > Features=0xbfebfbff > R,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT, > > TM,PBE> > >   > > Features2=0x9ce3bd > PR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT> > >   AMD Features=0x28100800 > >   AMD Features2=0x1 > >   VT-x: PAT,HLT,MTF,PAUSE,EPT,VPID > >   TSC: P-state invariant, performance statistics > > real memory  = 206158430208 (196608 MB) > > avail memory = 200380227584 (191097 MB) > > Event timer "LAPIC" quality 400 > Am I right that you only tested on stable/10 machines ? What you see > and I do not, might be due to the missed MFC of r296775, which commit > message claims that it fixes truncation in converions. > > Below is the commit merged to stable/10, could you test it ?  I do > not > have any stable/10 machine. > > Index: sys/kern/kern_event.c > =================================================================== > --- sys/kern/kern_event.c (revision 310169) > +++ sys/kern/kern_event.c (working copy) > @@ -556,34 +556,59 @@ knote_fork(struct knlist *list, int pid) >  #define NOTE_TIMER_PRECMASK (NOTE_SECONDS|NOTE_MSECONDS|NOTE_ > USECONDS| \ >   NOTE_NSECONDS) >   > -static __inline sbintime_t > +static sbintime_t >  timer2sbintime(intptr_t data, int flags) >  { > - sbintime_t modifier; >   > +        /* > +         * Macros for converting to the fractional second portion of > an > +         * sbintime_t using 64bit multiplication to improve > precision. > +         */ > +#define NS_TO_SBT(ns) (((ns) * (((uint64_t)1 << 63) / 500000000)) >> > 32) > +#define US_TO_SBT(us) (((us) * (((uint64_t)1 << 63) / 500000)) >> > 32) > +#define MS_TO_SBT(ms) (((ms) * (((uint64_t)1 << 63) / 500)) >> 32) >   switch (flags & NOTE_TIMER_PRECMASK) { >   case NOTE_SECONDS: > - modifier = SBT_1S; > - break; > +#ifdef __LP64__ > + if (data > (SBT_MAX / SBT_1S)) > + return SBT_MAX; > +#endif > + return ((sbintime_t)data << 32); >   case NOTE_MSECONDS: /* FALLTHROUGH */ >   case 0: > - modifier = SBT_1MS; > - break; > + if (data >= 1000) { > + int64_t secs = data / 1000; > +#ifdef __LP64__ > + if (secs > (SBT_MAX / SBT_1S)) > + return SBT_MAX; > +#endif > + return (secs << 32 | MS_TO_SBT(data % > 1000)); > + } > + return MS_TO_SBT(data); >   case NOTE_USECONDS: > - modifier = SBT_1US; > - break; > + if (data >= 1000000) { > + int64_t secs = data / 1000000; > +#ifdef __LP64__ > + if (secs > (SBT_MAX / SBT_1S)) > + return SBT_MAX; > +#endif > + return (secs << 32 | US_TO_SBT(data % > 1000000)); > + } > + return US_TO_SBT(data); >   case NOTE_NSECONDS: > - modifier = SBT_1NS; > + if (data >= 1000000000) { > + int64_t secs = data / 1000000000; > +#ifdef __LP64__ > + if (secs > (SBT_MAX / SBT_1S)) > + return SBT_MAX; > +#endif > + return (secs << 32 | US_TO_SBT(data % > 1000000000)); > + } > + return NS_TO_SBT(data); > + default: >   break; > - default: > - return (-1); >   } > - > -#ifdef __LP64__ > - if (data > SBT_MAX / modifier) > - return (SBT_MAX); > -#endif > - return (modifier * data); > + return (-1); >  } >   >  static void > Index: . > =================================================================== > --- . (revision 310169) > +++ . (working copy) > > Property changes on: . > ___________________________________________________________________ > Modified: svn:mergeinfo > ## -0,0 +0,1 ## >    Merged /head:r296775 > _______________________________________________ Yep, that was the solution.  After merging r296775 I now get: revolution > ./kevent_evfilt_timer  nsec = 10000009129 for 10000 loops of 1000000 nsec Thanks for tracking that down. -- Ian From owner-freebsd-hackers@freebsd.org Sat Dec 17 00:18:13 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F9AFC836BC for ; Sat, 17 Dec 2016 00:18:13 +0000 (UTC) (envelope-from vijju.singh@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 817941E30 for ; Sat, 17 Dec 2016 00:18:13 +0000 (UTC) (envelope-from vijju.singh@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 7DE76C836BB; Sat, 17 Dec 2016 00:18:13 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D96BC836BA for ; Sat, 17 Dec 2016 00:18:13 +0000 (UTC) (envelope-from vijju.singh@gmail.com) Received: from mail-ua0-x22a.google.com (mail-ua0-x22a.google.com [IPv6:2607:f8b0:400c:c08::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3EEBD1E2F for ; Sat, 17 Dec 2016 00:18:13 +0000 (UTC) (envelope-from vijju.singh@gmail.com) Received: by mail-ua0-x22a.google.com with SMTP id b35so25740255uaa.0 for ; Fri, 16 Dec 2016 16:18:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=ocRUOvrcQ+ByW1ZW6el5x/9KtXEz3cURqznlGazMSmE=; b=aPtJeDW/4j1PBAC1PDs6od08tOnH3/U+aE8iXO4GllKnl/rvO60eGC/bmPDWwImdVF 0/1+Tf5q5nbgQmxG03JIxMCzYGHMBQZfX+aoje3vMOdewJwWCtv2efX7XeEA+MNp4zPG r/1YFTiWvq05b0zGL5wpZ/oq2XdZHgzjOF4+vSf2nvgJ9jEK/3lwKdsg2g3MeIuv3Vv2 UeX0XzqgcI6sP4yOSvkrW7LktxNYBsRxW0t2Sw2q/WGOyRFW3+Rkt55WuhmkITG4tPwK yX/PdTA3AduWRzeZTpJIAZCWspjN1p6Fe/p3rDyiuGv4rrwg7jn2FrGNj/ObUjzHi3XV QpMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=ocRUOvrcQ+ByW1ZW6el5x/9KtXEz3cURqznlGazMSmE=; b=rUzBqJuHTQwJorfdQ7+6S3yb4z44aWs4nfHb6Fuda+hnKkaIFI9g5jxG9frUF9FaVg mwMlgtrArUkBa5w7oDfZrVLUxrRYfcWEdR4LG8V+Rsnq/9KSLI6Q+A5R+HIsGPtsIQTF iBbfS3MMcGx+NfPwEzuMdVaK2w46Wqyv4UlpN9U0ZXBcYdGNJgNALmnrTO2UEwh4Qosq hfyRwq0UaUU0T0v7q3vFjV4cQzZmVzELNHm35JZSwIwzF6w2c9LmB/7mn41S8gQ5FIs3 R7tczmEMA45BL7/KzMILjJG3vCvGmVFJCVECV8K21RKkQJ29r2wRksvTQvG/SBefOU7B /woA== X-Gm-Message-State: AIkVDXL4/1Nsj7R2mbYFcYO/2hj7VszhMoIAVt+N6fJrn9oYK6OOXySZjq6GCXeZ58lT3sDS4FCmY/uzQrZ9dQ== X-Received: by 10.176.85.158 with SMTP id v30mr2480018uaa.36.1481933892281; Fri, 16 Dec 2016 16:18:12 -0800 (PST) MIME-Version: 1.0 Received: by 10.31.92.7 with HTTP; Fri, 16 Dec 2016 16:18:11 -0800 (PST) From: Vijay Singh Date: Fri, 16 Dec 2016 16:18:11 -0800 Message-ID: Subject: OFED compilation To: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2016 00:18:13 -0000 Why does the OFED use no-depend in the makefile? -vijay From owner-freebsd-hackers@freebsd.org Sat Dec 17 05:22:06 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8A885C83856 for ; Sat, 17 Dec 2016 05:22:06 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 65CABDC; Sat, 17 Dec 2016 05:22:06 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (ppp121-45-230-194.lns20.per1.internode.on.net [121.45.230.194]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id uBH5M0OI059895 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 16 Dec 2016 21:22:03 -0800 (PST) (envelope-from julian@freebsd.org) Subject: Re: How to use sem_timedwait? To: Ian Lepore , Konstantin Belousov References: <20161214074228.zh6q5zya2gszw4g6@hal9000.meka.no-ip.org> <1481748960.1889.398.camel@freebsd.org> <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> <1481776532.1889.461.camel@freebsd.org> <20161215112050.GO94325@kib.kiev.ua> <1481817343.1889.467.camel@freebsd.org> <20161215171534.GV94325@kib.kiev.ua> <1481911613.1972.74.camel@freebsd.org> <20161216184553.GC94325@kib.kiev.ua> <1481931048.1343.2.camel@freebsd.org> Cc: freebsd-hackers@freebsd.org From: Julian Elischer Message-ID: <1b34cafe-c009-f939-fd2c-e31d45be2c9a@freebsd.org> Date: Sat, 17 Dec 2016 13:21:55 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <1481931048.1343.2.camel@freebsd.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2016 05:22:06 -0000 On 17/12/2016 7:30 AM, Ian Lepore wrote: > On Fri, 2016-12-16 at 20:45 +0200, Konstantin Belousov wrote: >> On Fri, Dec 16, 2016 at 11:06:53AM -0700, Ian Lepore wrote: >>> I tried LAPIC, HPET, and i8254 for eventtimer.hardware, there was >>> no >>> change with any of them. I tried turning periodic on, still no >>> change. [...] >>> =================================================================== >>> --- . (revision 310169) >>> +++ . (working copy) >>> >>> Property changes on: . >>> ___________________________________________________________________ >>> Modified: svn:mergeinfo >>> ## -0,0 +0,1 ## >>> Merged /head:r296775 >>> _______________________________________________ kib, are you going to merge that (if you havent already by the time you read this) ? > > Yep, that was the solution. After merging r296775 I now get: > > revolution > ./kevent_evfilt_timer > nsec = 10000009129 for 10000 loops of 1000000 nsec > > Thanks for tracking that down. > > -- Ian > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > From owner-freebsd-hackers@freebsd.org Sat Dec 17 09:00:21 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 61C79C82977 for ; Sat, 17 Dec 2016 09:00:21 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4F2BD958 for ; Sat, 17 Dec 2016 09:00:21 +0000 (UTC) (envelope-from hps@selasky.org) Received: by mailman.ysv.freebsd.org (Postfix) id 4E82CC82976; Sat, 17 Dec 2016 09:00:21 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4E1FDC82975 for ; Sat, 17 Dec 2016 09:00:21 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [88.99.82.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1A689957 for ; Sat, 17 Dec 2016 09:00:20 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id E8BB31FE157; Sat, 17 Dec 2016 10:00:12 +0100 (CET) Subject: Re: OFED compilation To: Vijay Singh , "freebsd-hackers@freebsd.org" References: From: Hans Petter Selasky Message-ID: <93fdb580-faa1-d25b-868f-19534dca1f9e@selasky.org> Date: Sat, 17 Dec 2016 09:59:45 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2016 09:00:21 -0000 On 12/17/16 01:18, Vijay Singh wrote: > Why does the OFED use no-depend in the makefile? > Which version of FreeBSD is this? I don't see it in 12-current. --HPS From owner-freebsd-hackers@freebsd.org Sat Dec 17 11:01:18 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1DD3C840D8 for ; Sat, 17 Dec 2016 11:01:18 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 251CDA1D; Sat, 17 Dec 2016 11:01:17 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBHB1CRd040201 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 17 Dec 2016 13:01:13 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBHB1CRd040201 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBHB1C9F040200; Sat, 17 Dec 2016 13:01:12 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 17 Dec 2016 13:01:12 +0200 From: Konstantin Belousov To: Julian Elischer Cc: Ian Lepore , freebsd-hackers@freebsd.org Subject: Re: How to use sem_timedwait? Message-ID: <20161217110112.GF94325@kib.kiev.ua> References: <20161215002906.mllorgvvuovbdtze@hal9000.meka.no-ip.org> <1481775511.1889.450.camel@freebsd.org> <1481776532.1889.461.camel@freebsd.org> <20161215112050.GO94325@kib.kiev.ua> <1481817343.1889.467.camel@freebsd.org> <20161215171534.GV94325@kib.kiev.ua> <1481911613.1972.74.camel@freebsd.org> <20161216184553.GC94325@kib.kiev.ua> <1481931048.1343.2.camel@freebsd.org> <1b34cafe-c009-f939-fd2c-e31d45be2c9a@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1b34cafe-c009-f939-fd2c-e31d45be2c9a@freebsd.org> User-Agent: Mutt/1.7.2 (2016-11-26) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2016 11:01:18 -0000 On Sat, Dec 17, 2016 at 01:21:55PM +0800, Julian Elischer wrote: > On 17/12/2016 7:30 AM, Ian Lepore wrote: > > On Fri, 2016-12-16 at 20:45 +0200, Konstantin Belousov wrote: > >> On Fri, Dec 16, 2016 at 11:06:53AM -0700, Ian Lepore wrote: > >>> I tried LAPIC, HPET, and i8254 for eventtimer.hardware, there was > >>> no > >>> change with any of them. I tried turning periodic on, still no > >>> change. > [...] > >>> =================================================================== > >>> --- . (revision 310169) > >>> +++ . (working copy) > >>> > >>> Property changes on: . > >>> ___________________________________________________________________ > >>> Modified: svn:mergeinfo > >>> ## -0,0 +0,1 ## > >>> Merged /head:r296775 > >>> _______________________________________________ > kib, are you going to merge that (if you havent already by the time > you read this) ? It was merged as r310183. From owner-freebsd-hackers@freebsd.org Sat Dec 17 17:34:16 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA618C85EC4 for ; Sat, 17 Dec 2016 17:34:16 +0000 (UTC) (envelope-from vijju.singh@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B86551C19 for ; Sat, 17 Dec 2016 17:34:16 +0000 (UTC) (envelope-from vijju.singh@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id B4AF1C85EC3; Sat, 17 Dec 2016 17:34:16 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4523C85EC2 for ; Sat, 17 Dec 2016 17:34:16 +0000 (UTC) (envelope-from vijju.singh@gmail.com) Received: from mail-ua0-x22f.google.com (mail-ua0-x22f.google.com [IPv6:2607:f8b0:400c:c08::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D9181C18 for ; Sat, 17 Dec 2016 17:34:16 +0000 (UTC) (envelope-from vijju.singh@gmail.com) Received: by mail-ua0-x22f.google.com with SMTP id y13so19913485uay.1 for ; Sat, 17 Dec 2016 09:34:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=LX1CRzQKcwOQ8sDmiJ14gWP/9Tx2TyjdLO1+W3ayCSs=; b=JIfuk2W7B/4DnN2EzlEVkQNl8x1T/9F0KTMdcF3PfP9LqtMvpKZvE9zcir5d32vEe9 AY4gIiuDThUm+sF/i92BNnHHrYQvIffB7GLsXuBGpATXYXZrKb1bA0z8cY4oVGqMaPPD pZFZxT5y/Bt04WLY4EucgSEPoPgZgsDY55lVpTjIpjYsM3jUxNfQt5Bd5Iwaf6xM91bB wl3LnR0pajIWDvYyY0rq2S0WJH6DuRZS9qqsOtHgOcbmO6wyLe5MkT/0BfzF3GBb58Uk XVTLL9IFC58gUAo5fX12rL9M0aoSypQI0j6rg6xRSks8todkmn+v1pfnE2ijwKlc1wLH V4LQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=LX1CRzQKcwOQ8sDmiJ14gWP/9Tx2TyjdLO1+W3ayCSs=; b=PR8I+6vf8Xq09Zql9/fI6mgJ9Mq/f3ML7WgkU4TxKGRSKsggVTenbVOfc7h2Iw4jj0 GMq0gcvl4DJOWN+7bphwSZDzuxBn1PQggUCNVuoCQY86WQDb8dmLQzm6xGj7lyhW5Ceb TrssXOYnuxMV9vVm8IKxhAMKt65ZB0C3+NZt09kHnkU25u5ShXKoPZ0aoFYT44eYsfTk GvIiSUoK/Xroutw85oMZQceQ+TTVo5MuUgd1eDZY2v7CN2vA/FT24TOSHy9ZqtcsjKEI sRsHxOF4SQpt3QiElD/OgNhpTeii26nE0phb0MR5Q9WFL3cmklbaxxBrKSCM8jlwlCC3 d4pA== X-Gm-Message-State: AIkVDXLkDpwkhMj8DNiZaCaAUUKg9Q/bdxl9SVKakgfBhdQQHKFzelPzz5g/sTO0KtP42vx20kcowWCu9vNT9w== X-Received: by 10.176.71.89 with SMTP id i25mr2108031uac.31.1481996055484; Sat, 17 Dec 2016 09:34:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.31.92.7 with HTTP; Sat, 17 Dec 2016 09:34:14 -0800 (PST) Received: by 10.31.92.7 with HTTP; Sat, 17 Dec 2016 09:34:14 -0800 (PST) In-Reply-To: <93fdb580-faa1-d25b-868f-19534dca1f9e@selasky.org> References: <93fdb580-faa1-d25b-868f-19534dca1f9e@selasky.org> From: Vijay Singh Date: Sat, 17 Dec 2016 09:34:14 -0800 Message-ID: Subject: Re: OFED compilation To: Hans Petter Selasky Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2016 17:34:16 -0000 I am looking at 9.2 and 10.3. On Dec 17, 2016 1:00 AM, "Hans Petter Selasky" wrote: > On 12/17/16 01:18, Vijay Singh wrote: > >> Why does the OFED use no-depend in the makefile? >> >> > Which version of FreeBSD is this? I don't see it in 12-current. > > --HPS > >