From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 6 16:45:20 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 88FB8387 for ; Mon, 6 Jan 2014 16:45:20 +0000 (UTC) Received: from nm3-vm0.bullet.mail.bf1.yahoo.com (nm3-vm0.bullet.mail.bf1.yahoo.com [98.139.212.154]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2ADCD144C for ; Mon, 6 Jan 2014 16:45:20 +0000 (UTC) Received: from [66.196.81.171] by nm3.bullet.mail.bf1.yahoo.com with NNFMP; 06 Jan 2014 16:39:38 -0000 Received: from [98.139.211.206] by tm17.bullet.mail.bf1.yahoo.com with NNFMP; 06 Jan 2014 16:39:38 -0000 Received: from [127.0.0.1] by smtp215.mail.bf1.yahoo.com with NNFMP; 06 Jan 2014 16:39:38 -0000 X-Yahoo-Newman-Id: 51286.23468.bm@smtp215.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: stIq8o8VM1mWu69qHy08uXrWnOX3vYDnc0822k4E3DXdZan aVIECd7bwQJZXU7NXqeXh0NqLFzqMYB8aFFHaGOUnpwz8ZX2VdvEsa9A1gGB KqnBo0I.dqJM9.ovAKBhdiAvzl0UqjZV4hRTp0L40QTPy.fRy3qu6oh3ogIZ jU6O_rYJ8rG71axxnSkkEI.GRDbVXcSC9gZBSqk0XI17fYmd55GlcBlR9QT4 j0N.MWIngPZmrKkFWCYxAIW3oMRTb557WUUqNAXyOA4myu3vp6q52iyeWB6W BFJH0DMx4.zFdAnobN62DTivoRYYBx2mWU_Uh21nPRXK54evAchJ6Yim2TSE 7Wj16EqunvO2nDWVBrwi52KxfaZyamhvb0yfIbIIGfcI59cup6kTmvAT_o_b JaObRw5AP0rc1JjADUQsE.1kAUOXrL38kLHzErq6YyjXLTZr9xRj.F7Z50jz LeAfUklP5VpoPIeyuIZIZU09ENyp7o37z6aQ9cL9tkqDQtGT0WUOBlorlcqr nFOOyfrlUNwUtn.Z2Lasxyr2JeQpoUXgAjHy_MgiNJIaWFiB38r5D9Qcoa.a sZUuELfZNaYubOHhwpI0VEZtybxiogULchL9eOfyIfjlUN5cOpvGtx6dLkCH P0uxqs65P0NPPXL5MeMbQ0kJnAg1T_O_s6I1Erp5rvgaq02c25cvg1vog2Mx BrwfQRaJuNENTRssiihucjIjcnFJVdA-- X-Yahoo-SMTP: xcjD0guswBAZaPPIbxpWwLcp9Unf X-Rocket-Received: from [192.168.0.102] (pfg@190.157.126.109 with plain [63.250.193.228]) by smtp215.mail.bf1.yahoo.com with SMTP; 06 Jan 2014 08:39:37 -0800 PST Message-ID: <52CADC46.1060200@FreeBSD.org> Date: Mon, 06 Jan 2014 11:39:34 -0500 From: Pedro Giffuni User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Jordan Hubbard Subject: GCD (was Re: svn commit: r260311 - in head/contrib: gcc gcc/cp gcc/doc gcclibs/include gcclibs/libiberty) References: <201401050043.s050hSMI089553@svn.freebsd.org> <20140105124557.5dd8395a@kalimero.tijl.coosemans.org> <52C985C7.9060406@FreeBSD.org> <9B829C42-1218-46F0-B3BF-D491A2F733B9@me.com> In-Reply-To: <9B829C42-1218-46F0-B3BF-D491A2F733B9@me.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: hackers@Freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jan 2014 16:45:20 -0000 (Time to move this thread to a list) On 06.01.2014 09:48, Jordan Hubbard wrote: > > On Jan 5, 2014, at 5:18 PM, Pedro Giffuni > wrote: > >> *Anyone working on a GCD-enabled version of grep or sort? :). > > Look at stdlib/FreeBSD/psort.c in OS X’s Libc > (http://www.opensource.apple.com/release/os-x-109/Libc-997.1.1) - > that’s the basis for the GCD-aware sort. > Ah , yes: http://opensource.apple.com/source/Libc/Libc-997.1.1/stdlib/FreeBSD/ I see some #include and the code is BSD licensed. > I don’t know if we got around to doing grep or not - I’d have to look. :) > Here is a nice video about GCD: http://youtu.be/nhbA9jZyYO4 There are many nice things to do (parallel patch/diff too), and we are one of the two OSs that support it. Pedro. From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 6 20:11:52 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 220B413D for ; Mon, 6 Jan 2014 20:11:52 +0000 (UTC) Received: from mail-ob0-x22c.google.com (mail-ob0-x22c.google.com [IPv6:2607:f8b0:4003:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DD9DB1637 for ; Mon, 6 Jan 2014 20:11:51 +0000 (UTC) Received: by mail-ob0-f172.google.com with SMTP id gq1so19051091obb.31 for ; Mon, 06 Jan 2014 12:11:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Hf9CyilTOTfor22GLTKOmAQvmw4VaN1iljQl2p24e/c=; b=cPt75b16i44vzO4FFXnM/weGFB01XzTkkYGW6Y5cb3VTZJSY1pm7LSCDr0CkrTZ5x6 2Q5WbZPU/jGm1iSojHzju61gTT8bOT4f4VXhnayhpzVEK/XstJ4Eno70XIh6K3ilFPbx b6T04OL+nydQDD7X4EwtonbX/ebiOu8FS8+yOGen7Moe7jN9A3MbGYVDlrQltg0rfWGK nw0eQpobc2lRiAs8YsQNChkstF6At0SX/Vf6weAv7KcaZF6tE5tx+vBffez4X4JtOf71 ki8oLQXxUtxA3wvfakQbl5/+X2TraUijytDY2vXtiJGXy+18BOdQLOXkZ2uRYky8rsqY PZig== MIME-Version: 1.0 X-Received: by 10.182.148.106 with SMTP id tr10mr1842912obb.65.1389039111085; Mon, 06 Jan 2014 12:11:51 -0800 (PST) Received: by 10.182.166.39 with HTTP; Mon, 6 Jan 2014 12:11:50 -0800 (PST) Date: Mon, 6 Jan 2014 12:11:50 -0800 Message-ID: Subject: Working on NUMA support From: Andrew Bates To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jan 2014 20:11:52 -0000 Hey all, My name is Andrew Bates, and I would like to take a bit of your time to talk about NUMA support. Supporting Non-Uniform Memory Access in FreeBSD is something that has been brought up in the past < http://freebsd.1045724.n5.nabble.com/NUMA-Support-is-there-in-FreeBSD-td486= 5200.html>. This is becoming increasingly important now that multiprocessor systems are an expanding technology, thus performance is scaling in terms of cpu count, rather than just clock rate. There is a great opportunity here to optimize performance. After being asked to look into this by the EMC Isilon Storage Division, myself and a few colleagues advised by Andrew Pilloud and Jeff Roberson would like to propose APIs to handle basic memory allocation/management to specific NUMA domains. What we have devised so far consists of two levels. First there are the KPIs, to expose NUMA functionality at a thread level of domain affinity. Secondly, there would be a userspace/interface to take advantage of the proposed APIs, thus giving users the capability to make their applications NUMA-aware. We took the time to look into how many other systems (Linux, Macintosh, Solaris, Windows) already approach this problem, so there are some aspects of our solution that are similar to how Linux and Solaris handle NUMA. Unlike Linux libnuma, we are only proposing a few additions and a minimal library that can easily be expanded later to suit users=92 needs. KISS in mind, we came up with the following KPI prototypes (freebsdnuma.h) to uncover NUMA in a usable fashion: - cpuset_get_memory_affinity() - cpuset_set_memory_affinity() - move_pages() - migrate_pages() - get_numa_cpus() - get_numa_weights() Then to the second part, we have the following userspace API prototypes (numanor.h) for our interface and testing purposes: - is_numa_available() - set_thread_on_domain() - set_memory_policy() - move_thread() In much much more detail, you can learn more about these prototypes, this project, view our progress, track along, and give input on our github repo < https://github.com/andrewbates09/freebsd-numa > or simply via email. This repo currently includes fully commented prototypes (like a mini man page) and will later include additions to the project. If anyone has any comments, suggestions, concerns, quandaries, or just general thoughts please feel free to contact us, as we would love to hear your input! The Leaders: Sakire Arslan Ay, Andrew Pilloud, Jeff Roberson The Team: Andrew Bates, Joshua Clark, Alex Schuldberg, Dustin Walker --=20 V/Respectfully, Andrew M Bates From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 6 21:53:39 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 86C2DDFF for ; Mon, 6 Jan 2014 21:53:39 +0000 (UTC) Received: from mta05.bitpro.no (mta05.bitpro.no [92.42.64.202]) by mx1.freebsd.org (Postfix) with ESMTP id 414C21E0A for ; Mon, 6 Jan 2014 21:53:38 +0000 (UTC) Received: from mail.lockless.no (mail.lockless.no [46.29.221.38]) by mta05.bitpro.no (Postfix) with ESMTPS id 639A317FC62; Mon, 6 Jan 2014 22:53:36 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by mail.lockless.no (Postfix) with ESMTP id 30DFB8F22A5; Mon, 6 Jan 2014 22:54:23 +0100 (CET) X-Virus-Scanned: by amavisd-new-2.6.4 (20090625) (Debian) at lockless.no Received: from mail.lockless.no ([127.0.0.1]) by localhost (mail.lockless.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NZhHYCk58WZP; Mon, 6 Jan 2014 22:54:22 +0100 (CET) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) by mail.lockless.no (Postfix) with ESMTPSA id 8A6818F2166; Mon, 6 Jan 2014 22:54:22 +0100 (CET) Message-ID: <52CB2627.1000003@bitfrost.no> Date: Mon, 06 Jan 2014 22:54:47 +0100 From: Hans Petter Selasky Organization: Bitfrost A/S User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: by , "freebsd-hackers@freebsd.org" Subject: Re: Strange keyboard mistake References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jan 2014 21:53:39 -0000 On 01/03/14 05:00, by wrote: > Hi, > I got a very strange problem. > I got another keyboard for my laptop, everything goes well for days, but today, for some reasons when in csh environment, i got my new keyboard off my laptop's USB port, and just a few minutes later, after i put it back, keyboard got a mistake. > For example, when i type 'b', it became a "smile face", and other keys became other strange symbols too! > What is the most strange is that my original keyboard on my laptop became the same! > I got no idea how to do, so i hit Ctrl+Alt+Delete to reboot, after reboot, everything became normal. > Does anyone got any ideas about this strange behavior? Or if i do not want to reboot, what should i do when i encounter this situation again. > By the way, i use FreeBSD 8.4 RELEASE and my new keyboard is Logitech K310. > Thanks. > ----by Hi, Are you sure you didn't press any LOCK keys, like NUMLOCK, CAPSLOCK or SCROLL LOCK when this happened? "usbdump -i usbusX -f Y -vvv -s 65536" will dump the actual traffic towards the USB device. Maybe your USB keyboard has some "bugs" in it. --HPS From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 00:51:15 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5BD68FB; Tue, 7 Jan 2014 00:51:15 +0000 (UTC) Received: from mail-ee0-x22d.google.com (mail-ee0-x22d.google.com [IPv6:2a00:1450:4013:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E1F201E51; Tue, 7 Jan 2014 00:51:14 +0000 (UTC) Received: by mail-ee0-f45.google.com with SMTP id d49so8134661eek.32 for ; Mon, 06 Jan 2014 16:51:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=BdWKlxaOo5K/xSI5/LaSYD3PDKYCFs1xm2eQpAB4udg=; b=HeEI00CFOPWtPAmpXCHiD1AcMOwNlbssOiQiXvcOZq52pmqQF1+RJ2MsxqFOyaFCPE DzBw9mIfVGxiylGAYljvgKV92RES3r9J0D+Zd385lmFM9J5SbjCEs0iyRJLfk0tTl9YA PzoiLPBGSEvvb/gveSsqG6f0kXHCW/vSKL4/YQJ6Su5DoeS+2YHIm2vYbWKNOYKjm1l8 79DbEbRp/SPXSOW/AqkCNaU6jB++pZECG5s67xM1DewMC82fL3GA7F/5SXNrwBHPWtij eu+b8DZ5/a/kvZlwg4dkQeURGmhcqq1Dxp4n8JnZqKU2CO45APZ3pYWFNz6lUjo+5ddo Cdeg== X-Received: by 10.15.24.142 with SMTP id j14mr38148671eeu.52.1389055873259; Mon, 06 Jan 2014 16:51:13 -0800 (PST) Received: from mavbook.mavhome.dp.ua ([134.249.139.101]) by mx.google.com with ESMTPSA id a51sm175388935eeh.8.2014.01.06.16.51.10 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 06 Jan 2014 16:51:11 -0800 (PST) Sender: Alexander Motin Message-ID: <52CB4F7D.2080909@FreeBSD.org> Date: Tue, 07 Jan 2014 02:51:09 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: hackers@freebsd.org Subject: UMA caches draining Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 00:51:15 -0000 Hi. I have some questions about memory allocation. At this moment our UMA never returns freed memory back to the system until it is explicitly asked by pageout daemon via uma_reclaim() call, that happens only when system is quite low on memory. How does that coexist with buffer cache and other consumers? Will, for example, buffer cache allocate buffers and be functional when most of system's memory uselessly consumed by UMA caches? Is there some design how that supposed to work? I've made an experimental patch for UMA (http://people.freebsd.org/~mav/drain_unused.patch) to make it every 20 seconds return back to the system cached memory, unused for the last 20 seconds. Algorithm is quite simple and patch seems like working, but I am not sure whether I am approaching problem from the right side. Any thoughts? -- Alexander Motin From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 05:48:38 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E5FDA775; Tue, 7 Jan 2014 05:48:38 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 71AF91317; Tue, 7 Jan 2014 05:48:38 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id s075mQWN061811; Tue, 7 Jan 2014 07:48:26 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s075mQWN061811 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id s075mPuf061810; Tue, 7 Jan 2014 07:48:25 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 7 Jan 2014 07:48:25 +0200 From: Konstantin Belousov To: Alexander Motin Subject: Re: UMA caches draining Message-ID: <20140107054825.GI59496@kib.kiev.ua> References: <52CB4F7D.2080909@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="7urMzS8W4fOs217t" Content-Disposition: inline In-Reply-To: <52CB4F7D.2080909@FreeBSD.org> User-Agent: Mutt/1.5.22 (2013-10-16) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 05:48:39 -0000 --7urMzS8W4fOs217t Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jan 07, 2014 at 02:51:09AM +0200, Alexander Motin wrote: > Hi. >=20 > I have some questions about memory allocation. At this moment our UMA=20 > never returns freed memory back to the system until it is explicitly=20 > asked by pageout daemon via uma_reclaim() call, that happens only when=20 > system is quite low on memory. How does that coexist with buffer cache=20 > and other consumers? Will, for example, buffer cache allocate buffers=20 > and be functional when most of system's memory uselessly consumed by UMA= =20 > caches? Is there some design how that supposed to work? Allocation of the pages which consitute a new buffer creates the pressure and causes pagedaemon wakeup if amount of free pages is too low. Look at the vm_page_grab() call in allocbuf(). Also note that buffer cache is not shrinked in response to the low memory events, and buffers pages are excluded from the page daemon scans since pages are wired. >=20 > I've made an experimental patch for UMA=20 > (http://people.freebsd.org/~mav/drain_unused.patch) to make it every 20= =20 > seconds return back to the system cached memory, unused for the last 20= =20 > seconds. Algorithm is quite simple and patch seems like working, but I=20 > am not sure whether I am approaching problem from the right side. Any=20 > thoughts? >=20 > --=20 > Alexander Motin --7urMzS8W4fOs217t Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJSy5UoAAoJEJDCuSvBvK1BIwsP/0yzFSF1REoZz7us7XQr2XRQ EriCJDyk4XUCFVCZOYgYAHu+z2iRfQ1NdnYY3FXm3DVNj/DSrPPO9OPJq+YgB/6V eBL1B6haZi3PqdaaRwhyhQaV+LuEZBItyrBbMT0Xfu4Myd5iHHe38AXgwd2+rRqn 9KY0ipdyfQWzp15+hzRmWnPe/7ziR6XjEO6mGkfDNcL0ZgDAUCRqjqJPp7WNSUHG pd3EybTGylLOwubWHhxlpvYqIYP1BXRHbJI23Tmr4gXucsmBglEF5sL6RCxYrHao W1xMbuZxyfnXF6scrSHq0E5/Mq1AS77zMo+joz5YfLNSDn1af8KkIlHvpdLatFMt p4caq4lcOLlYPjZg6O0DTOoOm/bWZ3MMkocY3+phDfz4dg0wlN9Mwi4KMjBPx8nV 7ZqHo9fq+IDAi1/+hmxxZXf2zrs4Z+xN9zwaXfO3yvADmrzkUCykK22Batb+2m4S pB3r2KVeru3sjxTL+QCBNH/PxYqLZKDuO96U3HJgxePsaOG1t+GGTaXWfxcMIH2X hmPJeEPf7HZ2Wl7s1tnotm3SDmY5lH/r3A5za7jQwtAZpm0N3NNZsKmOcS57lllj LlXDcBUtEoy0QEaoj5ZYNwyB8wzdVW8GLeq9tz+2RBFt4Po6gmR0UJKUopo8dBY9 zvgIVtiXfKQyZdSr38Gk =I3c7 -----END PGP SIGNATURE----- --7urMzS8W4fOs217t-- From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 05:50:35 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BDD49891; Tue, 7 Jan 2014 05:50:35 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 429E11328; Tue, 7 Jan 2014 05:50:35 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id s075oUX9062545; Tue, 7 Jan 2014 07:50:30 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s075oUX9062545 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id s075oUd1062544; Tue, 7 Jan 2014 07:50:30 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 7 Jan 2014 07:50:30 +0200 From: Konstantin Belousov To: Oleg Bulyzhin Subject: Re: atomic_load_acq @ i386/amd64 Message-ID: <20140107055030.GJ59496@kib.kiev.ua> References: <20140103205159.GA99722@lath.rinet.ru> <20140104172923.GY59496@kib.kiev.ua> <20140104232910.GA12331@lath.rinet.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="MEhAVL6rS7bF26GE" Content-Disposition: inline In-Reply-To: <20140104232910.GA12331@lath.rinet.ru> User-Agent: Mutt/1.5.22 (2013-10-16) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 05:50:35 -0000 --MEhAVL6rS7bF26GE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jan 05, 2014 at 03:29:10AM +0400, Oleg Bulyzhin wrote: > On Sat, Jan 04, 2014 at 07:29:23PM +0200, Konstantin Belousov wrote: > > On Sat, Jan 04, 2014 at 12:51:59AM +0400, Oleg Bulyzhin wrote: > > >=20 > > > Hello. > > >=20 > > > I've got a question: why atomic_load_acq_* implemented on i386/amd64 = archs > > > with locked cmpxchg instruction? Comment about this > > > (in /sys/(amd64|i386)/include/atomic.h) looks wrong for me. I believe > > > acquire/release semantics does not require StoreLoad barrier so simpl= e aligned > > > load should be enough. (because acquire/release semantics does not gu= arantee > > > sequential consistency). > >=20 > > You did not explicitely wrote which statement in the comment is false, = in > > your opinion. >=20 > >=20 > > FreeBSD assumes a property of _acq/_rel stuff which is sometimes called > > 'total lock ordering'. It is indeed sort of sequential consistency, but > > only for atomic+membar ops. Would atomic_load_acq() implemented as pla= in > > load, it can pass stores, in particular stores from the _rel op, which > > breaks the guarantee. > >=20 > > For x86, there are indeed two possible schemes for implementing critical > > section, one is lock cmpxchg for get(), and plain store for release(), > > which is what we use. Another is plain load for get(), and xchg for > > release(). Then, the load_acq() must be adopted to not break the acq/r= el > > consistency, and since we use plain store for release(), load_acq must > > use serialing instruction. >=20 > Perhaps i was not clear enough, i'm talking about this one: > "However, loads may pass stores, so for atomic_load_acq we have to > ensure a Store/Load barrier to do the load in SMP kernels." >=20 > As far as i know acquire/release semantics guarantees following: > if we have this code > > _acq > > _rel > >=20 > following statements are true: > 1) cannot leave (due to reordering) acq/rel block > 2) may leak past _acq=20 > 3) may leak before _rel > So neither _acq nor _rel requires full membar. I.e. > op_acq is: > > up reordering is prohibited> > op_rel is: > down reordering is prohibited> > >=20 > Intel documentation says about only thing (for simple load/stores) can be > reordered: "Reads may be reordered with older writes to different locatio= ns > but not with older writes to the same location." >=20 > So, if older store can pass our load_acq() it would not break requirement= s. > And i do not understand how load op from load_acq() can pass store op from > store_rel(), intel doc says: "Writes are not reordered with older reads".= =20 Please re-read what I wrote above about 'total lock ordering'. >=20 > Well, while writing this email i realized what is disturbing me: it's ato= mic(9) > "Multiple Processors" section. It claims atomics are not atomic in common= MP > case and says atomics are atomic @i386. It looks strange for me: > 1) i guess it's not "atomic" even for i386/MP without proper membar pairi= ng. > 2) if we have acq/rel modifiers for atomics why we cannot guarantee "atom= icity" > for any MP arch? >=20 > P.S. please correct me if i'm wrong in my statements, i'm spending my new= year > holidays for ignorance elimination. ;) I do not know what do you mean by 'not atomic'. --MEhAVL6rS7bF26GE Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJSy5WlAAoJEJDCuSvBvK1B/r8P/i0sYZOX4UtHPAZ9/dU8KqBi +nxMCdLFO3wlunp38okA2Osu0jg4zb8395FUbnXDKcUXg80EPGfRq62s39Pp45LS 0WJvmJXyxpSfugBELrS+wipPqFIcefshjVVSXVSxdZO+KYxM466JwtbwgVpFQQTI Lt3QK/r8mNeueu7VlLHWWuTtNLkyaKSxu9x9fwek46ozeoB/wWvzjdoE6pDyDXDC lDNHtqjNa897iIVrww/Wo56P4yi8yuGMdKdlBlhnNQDHQxTn+j5eJsPGkGTwPeDr CjJ3hp3qi/1gm6xROAKSiBdLathhVaWl2kbxux6Xxm+7ExlOHyYslFL53TcU7NUl s8wcIauRuDMtAJg8zWCAmailrk+VhJ6PVNmXwG5SYaLbx9g+KDIKXwVLkBVEaNj/ zj08GqvdimzF7LQAbPrx/7J1M2Dm2Z/Eb/eD+5YAt1YR53QdkW6yqIPou0QbEutv rTXLCnWGpawQNAo5ys+OlKTH++ndqIr7q1XCjXns4to+V2+yMA+EvtpBaSeTapt5 /yJDln0YNlTlJkRtbdS+WH9XtE8dBEfFjzHEDNil7ErJjloXYBEqfoTiGfkz8o5g ba6QXY3E9w4JZco70BgofP9+4S724VTEw3JIMcPe3xFtnPCoQMoiit4sR5nHxtI+ awJkRKHJpB//LIQAC4RW =ZbcN -----END PGP SIGNATURE----- --MEhAVL6rS7bF26GE-- From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 09:44:29 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B28D34C5 for ; Tue, 7 Jan 2014 09:44:29 +0000 (UTC) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4806713D7 for ; Tue, 7 Jan 2014 09:44:28 +0000 (UTC) Received: by mail-ee0-f54.google.com with SMTP id e51so7158355eek.27 for ; Tue, 07 Jan 2014 01:43:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=tDXf+ez4FONTJwRdryZbJjtIWhjjGIwDq+GRP5TDMxo=; b=nrOK9SItZaHCu2Rp881VfAHdtrZMnFUhecbAmI/5Nxdc11Yv+2t1g2zWPR+jO0X994 UdEPNtjJh2pnLxPwxZGtCy5T4wAE/7mt2AzcCCYlfrNfCxn9Iwy6X+ecJG4xM/GhKUdQ mZE1Hskvtpp47JDL1KRsoxxXhczjA/e7egktvILI5mNeD+WEpI1NXqqWmQJ1AEKtEKtw rhBIWazw0gMob8tIrvBh03DpM5lIShO8lo4IbC+66UHvj0pn0vP6AzQ/MgeuOPA3G4Xg 4SzT5+Cn2zalyHHqqm+Ao0+0MYC/LOe8IzfzC8U30g75jlnrUdxjpEfToAKlYjGUgba2 znjQ== X-Received: by 10.14.172.130 with SMTP id t2mr20083620eel.68.1389087826305; Tue, 07 Jan 2014 01:43:46 -0800 (PST) Received: from mavbook.mavhome.dp.ua ([134.249.139.101]) by mx.google.com with ESMTPSA id h3sm178380265eem.15.2014.01.07.01.43.44 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 07 Jan 2014 01:43:45 -0800 (PST) Sender: Alexander Motin Message-ID: <52CBCC4F.8020900@FreeBSD.org> Date: Tue, 07 Jan 2014 11:43:43 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: UMA caches draining References: <52CB4F7D.2080909@FreeBSD.org> <20140107054825.GI59496@kib.kiev.ua> In-Reply-To: <20140107054825.GI59496@kib.kiev.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 09:44:29 -0000 On 07.01.2014 07:48, Konstantin Belousov wrote: > On Tue, Jan 07, 2014 at 02:51:09AM +0200, Alexander Motin wrote: >> I have some questions about memory allocation. At this moment our UMA >> never returns freed memory back to the system until it is explicitly >> asked by pageout daemon via uma_reclaim() call, that happens only when >> system is quite low on memory. How does that coexist with buffer cache >> and other consumers? Will, for example, buffer cache allocate buffers >> and be functional when most of system's memory uselessly consumed by UMA >> caches? Is there some design how that supposed to work? > Allocation of the pages which consitute a new buffer creates the pressure > and causes pagedaemon wakeup if amount of free pages is too low. Look > at the vm_page_grab() call in allocbuf(). Also note that buffer cache > is not shrinked in response to the low memory events, and buffers pages > are excluded from the page daemon scans since pages are wired. Thanks. I indeed can't see respective vm_lowmem handler. But how does it adapt then? It should have some sort of back pressure. And since it can't know about UMA internals, it should probably just see that system is getting low on physical memory. Won't it shrink itself first in such case before pagedaemon start its reclaimage? When vm_lowmem condition finally fire, it will purge different data from different subsystems, potentially still usable. UMA caches though have no valid data, only an allocation optimization. Shouldn't they be freed first somehow, at least an unused part, as in my patch? Also I guess having more really free memory should make M_NOWAIT allocations to fail less often. >> I've made an experimental patch for UMA >> (http://people.freebsd.org/~mav/drain_unused.patch) to make it every 20 >> seconds return back to the system cached memory, unused for the last 20 >> seconds. Algorithm is quite simple and patch seems like working, but I >> am not sure whether I am approaching problem from the right side. Any >> thoughts? -- Alexander Motin From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 16:10:05 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6F601E37 for ; Tue, 7 Jan 2014 16:10:05 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2AC20154D for ; Tue, 7 Jan 2014 16:10:04 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1W0ZE9-0002fF-5c for freebsd-hackers@freebsd.org; Tue, 07 Jan 2014 17:10:01 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 07 Jan 2014 17:10:01 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 07 Jan 2014 17:10:01 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Subject: Continual benchmarking / regression testing? Date: Tue, 07 Jan 2014 17:09:50 +0100 Lines: 33 Message-ID: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WfvkWG4iRAO4KTlf7xjbSJi5j5aI3VFNP" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 X-Enigmail-Version: 1.5.2 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 16:10:05 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --WfvkWG4iRAO4KTlf7xjbSJi5j5aI3VFNP Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hello, Is someone working on a contitual benchmarking / regression testing project for FreeBSD? I seem to recall there was a post several months ago but I can't find it. There was talk about it at the last BSDCan, so if nobody's working on it, I'll try to revive the project in time for the next BSDCan :D --WfvkWG4iRAO4KTlf7xjbSJi5j5aI3VFNP Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iKYEARECAGYFAlLMJs9fFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDYxNDE4MkQ3ODMwNDAwMDJFRUIzNDhFNUZE MDhENTA2M0RGRjFEMkMACgkQ/QjVBj3/HSw4LACggw8XqjH+lAUWufystFwBm5HD 8+MAoIAYlndClArjOUw9MrnTrueR5VlH =Up5c -----END PGP SIGNATURE----- --WfvkWG4iRAO4KTlf7xjbSJi5j5aI3VFNP-- From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 16:11:51 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BE845BB for ; Tue, 7 Jan 2014 16:11:51 +0000 (UTC) Received: from mail-lb0-f177.google.com (mail-lb0-f177.google.com [209.85.217.177]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 47A6815D9 for ; Tue, 7 Jan 2014 16:11:51 +0000 (UTC) Received: by mail-lb0-f177.google.com with SMTP id q8so422280lbi.36 for ; Tue, 07 Jan 2014 08:11:42 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=pIozTXk06OIXr5GGXhMz/vpetMKejc0wniIlte6sF7c=; b=MYBTYFeIXUNBe9gLX1zaKUyFteSu5ikyPGKT6bT8bhR23pTDFzpeaWfE31wuZz74ML fAatN1DgMlYKWjc5W7bO2Y6AQXIrTC8vaSR7EWD/8H70zXfJsJN3WVe/SzQ8BSlO3D5I aMgeG3m95n/tk8z+39Ep1m/MNqEGihrvXa//ykyGijy4+8vARVWbB5BwZ6BweG2U4Wly pxveKLtANWw0YN7tgeRWnnQ99bIcpmqtG+NO+tKipx/tL5sWrmbJsuwU8WXr8O7nsAOb w235d5vmXN2XnALRRInCfQIghg1fJQABPyrH9CyluJbnakCalaIvdQyJ9KhNVMgWiqKt e/hQ== X-Gm-Message-State: ALoCoQnW5qotvyomZ2UhCia5VlSm/H9Y/IEdn8dgjcRgT0cioA86teBFzWbvp334IRInjYH1xdrT X-Received: by 10.152.234.75 with SMTP id uc11mr3899109lac.30.1389111102519; Tue, 07 Jan 2014 08:11:42 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.129.101 with HTTP; Tue, 7 Jan 2014 08:11:22 -0800 (PST) X-Originating-IP: [2620:0:1040:406:9098:ad1a:255:209e] In-Reply-To: References: From: Julio Merino Date: Tue, 7 Jan 2014 16:11:22 +0000 Message-ID: Subject: Re: Continual benchmarking / regression testing? To: Ivan Voras Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 16:11:51 -0000 On Tue, Jan 7, 2014 at 4:09 PM, Ivan Voras wrote: > Hello, > > Is someone working on a contitual benchmarking / regression testing > project for FreeBSD? I seem to recall there was a post several months > ago but I can't find it. See http://wiki.freebsd.org/TestSuite for the current efforts. -- Julio Merino / @jmmv From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 16:18:00 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ABA2D2D3 for ; Tue, 7 Jan 2014 16:18:00 +0000 (UTC) Received: from mail-vb0-x235.google.com (mail-vb0-x235.google.com [IPv6:2607:f8b0:400c:c02::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6933F1619 for ; Tue, 7 Jan 2014 16:18:00 +0000 (UTC) Received: by mail-vb0-f53.google.com with SMTP id o19so249462vbm.40 for ; Tue, 07 Jan 2014 08:17:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=tc6MO78b8iwhBbYPL6Oy1h5rgLMdSDwswL99Ll+fzAA=; b=H5fU0T4vI8vwX0mNHvibz2woHOGfcD1++ZU/zrH5b+SmXo8VkPwpNgJXW7DNT9pZlc J3w3y5LLlSG7NlAKBiqShIi2I061auS/lo95J3sXGycGI/P9KyAPKV9jawkK4iqGeCN2 MiUjPG2oZyOgH83T6o6ApNv9A37HEBMTEeMbYpydt2S1TpArzENy9k2eyLbSd/H9K5sU UK0agMDwayd46NeyvWvpvkU6N90ZQm1TZUNtJkZRSRDct07WyMYxjY0Ztc/zzmsVz2pY bsU7ic22xLvIbvaszFSCDHsKMjKEvhWM04ktaJ8UYdmvFCuM7ARo4eLNR6lDn9WPKJ2Y Hfow== X-Received: by 10.52.109.105 with SMTP id hr9mr2664803vdb.71.1389111479451; Tue, 07 Jan 2014 08:17:59 -0800 (PST) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.58.134.98 with HTTP; Tue, 7 Jan 2014 08:17:19 -0800 (PST) In-Reply-To: References: From: Ivan Voras Date: Tue, 7 Jan 2014 17:17:19 +0100 X-Google-Sender-Auth: FdH6DkUf371NHxjx44mWSJkDWZk Message-ID: Subject: Re: Continual benchmarking / regression testing? To: Julio Merino Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 16:18:00 -0000 On 7 January 2014 17:11, Julio Merino wrote: > On Tue, Jan 7, 2014 at 4:09 PM, Ivan Voras wrote: >> Hello, >> >> Is someone working on a contitual benchmarking / regression testing >> project for FreeBSD? I seem to recall there was a post several months >> ago but I can't find it. > > See http://wiki.freebsd.org/TestSuite for the current efforts. Ok, by looking at the wiki page and http://kyua1.nyi.freebsd.org/ (only stable/10 is available btw), it looks like this effort is centered on correctness, not performance benchmarking? Is Kyua easily adaptable to include graphs and other more visually attractive presentation types? From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 16:16:47 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90CEF2C3 for ; Tue, 7 Jan 2014 16:16:47 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 19B7E1612 for ; Tue, 7 Jan 2014 16:16:46 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id er20so265990lab.8 for ; Tue, 07 Jan 2014 08:16:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:content-transfer-encoding:subject:message-id:date :to:mime-version; bh=vqwWgFCCfKNiC+wFn/B/FHbis7LvPWgEnHR0/WHqtus=; b=pQ5p9uqUNgCNQYPd78DTmiaueSgnw2pyAoaBCeyqReFO6vZRUbuOfc3bk6mB0gljxk LgKfJIluM3nIuqYTPygvW9MfxIedLvDxxuelejQmYO43MGrK2wdT2Fx9FOSKOigJyiJU yf9X+G47o8HuBMKBnZqlVxfaX+EpZQCPT0ukSDo0wIj3QcV0+qTBvioSEQj7KUXgpmnP gGHMFdJipbkPP3taqAS3B3z1Yn1PJQ3/bI5FNoCgxDwz31OpG+jF0IGQ4lpoT8hQTA18 TByI9y6zUZMMdVyc/HTcF9B18iPYfvEqyeRMUn01UOyKKz+Bxxnjc81y63ND5wCZJA9d izDw== X-Received: by 10.152.28.230 with SMTP id e6mr46945956lah.3.1389111404963; Tue, 07 Jan 2014 08:16:44 -0800 (PST) Received: from [10.0.1.20] ([176.193.50.79]) by mx.google.com with ESMTPSA id e6sm45226700lbs.3.2014.01.07.08.16.43 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 07 Jan 2014 08:16:43 -0800 (PST) From: Dmitry Sivachenko Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: FreeBSD-10 and processes swapping out for no reason Message-Id: Date: Tue, 7 Jan 2014 20:16:42 +0400 To: hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) X-Mailer: Apple Mail (2.1827) X-Mailman-Approved-At: Tue, 07 Jan 2014 16:48:58 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 16:16:47 -0000 Hello! Recently I updated several of my servers from 9.2-STABLE 10.0: 10.0-PRERELEASE FreeBSD 10.0-PRERELEASE #0 r259661 Workload did not change at all, same programs, same load, same hardware. I noticed that some of the processes on 10.0 boxes are marked as swapped = out in top(1) output: 1436 root 1 43 0 16524K 0K nanslp 14 1:14 0.00% = 1381 smmsp 1 20 0 23988K 0K pause 18 0:04 0.00% = ps also shows them as swapped out (W as second character in state = field): 1381 - IWs 0:00.00 sendmail: Queue runner@00:30:00 for = /var/spool/clie 1436 - IWs 0:00.00 /usr/sbin/cron -s 80231 - IWs 0:00.00 /usr/local/sbin/collectdmon -c = /usr/local/sbin/coll 99348 1 IWs 0:00.00 -csh (csh) These machines have enought RAM and that never happened on 9.2-STABLE. I turned off swap: # swapinfo Device 1K-blocks Used Avail Capacity # And state of these processes did not change: still second characted in = state field of ps output is W, and top shows them in angles. How should I treat that? Thanks.= From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 17:01:39 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 91726B91; Tue, 7 Jan 2014 17:01:39 +0000 (UTC) Received: from mail-ve0-x230.google.com (mail-ve0-x230.google.com [IPv6:2607:f8b0:400c:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3AA371A1D; Tue, 7 Jan 2014 17:01:39 +0000 (UTC) Received: by mail-ve0-f176.google.com with SMTP id oz11so331769veb.21 for ; Tue, 07 Jan 2014 09:01:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=uBQiX9gtxcSNPeR0lpSPptryT21zewLXMGAO7peDiiM=; b=l9wtF0cX5LE8UvL3mZok37cbkKfLP3bkEPzM6HmT7N7IndGPa9EFTGYR0o0OgGnvdC U9AIfSHQHUC6+Ff7m8BhRVkQyjophkuTY54JjV2qI3HCrmkufCEIrWXR8jADNKB05Zle PVus8d5qtAJ16ZapbJ5JpAose6tZqd3mLah8cBOqCraKrKKHQDL+ZxZ6k1j5lajzFXoT AzEoyTnr3fqhZ2Pkf8ADnD/cQDj5ONYgf0HioL7YGqToIxkXVScBCXEW2eVLzS5pfaBp dzubIeSrTPstvP5NYN69l1HhE8vzmcZliNYrpwNP7rz/3aDt6ah9SNJq9lxW+oFqQVIy jmCw== MIME-Version: 1.0 X-Received: by 10.52.157.68 with SMTP id wk4mr9308191vdb.19.1389114098215; Tue, 07 Jan 2014 09:01:38 -0800 (PST) Sender: asomers@gmail.com Received: by 10.58.57.163 with HTTP; Tue, 7 Jan 2014 09:01:38 -0800 (PST) In-Reply-To: References: Date: Tue, 7 Jan 2014 10:01:38 -0700 X-Google-Sender-Auth: 6U-crYLiw7k8-P2Pwa0VZROfcRI Message-ID: Subject: Re: Continual benchmarking / regression testing? From: Alan Somers To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" , Ivan Voras X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 17:01:39 -0000 On Tue, Jan 7, 2014 at 9:11 AM, Julio Merino wrote: > On Tue, Jan 7, 2014 at 4:09 PM, Ivan Voras wrote: >> Hello, >> >> Is someone working on a contitual benchmarking / regression testing >> project for FreeBSD? I seem to recall there was a post several months >> ago but I can't find it. > > See http://wiki.freebsd.org/TestSuite for the current efforts. I think that Kyua is less than ideal for benchmarking. It could be extended, but there are two fundamental differences between a test framework and a benchmark framework: 1) Benchmarks are slow. Not only that, but they usually come with a bewildering array of options (file size, I/O size, etc) that exponentially increase the time required to do a comprehensive run of all available tests with all available options. So, you don't want to run all of the benchmarks all of the time. In contrast, tests are usually fast, and you usually want to run all of them all of the time. 2) Tests usually have a binary output. Did it pass or didn't it? Kyua has a few other possible outcomes (expected failure, skipped, broken), but it's still a short list. In contrast, benchmarks usually have a variable output, expressed as one (or more) real numbers. IMO, the extensions that would be required for Kyua to function as a benchmark framework would be too intrusive; they would make it more difficult to maintain Kyua's role as a test framework, and add nothing to Kyua's testing abilities. I think that a separate benchmarking framework would be better. The best benchmarking framework that I know of is the Phoronix Test Suite (http://www.phoronix-test-suite.com/) . Its cross-platform, it has a decent report generator, including a public list of results at http://openbenchmarking.org/, and a huge library of benchmark programs. However, it has several drawbacks. Many of the benchmark programs are of poor quality IMHO; they seems like that get committed without sufficient analysis to make sure that they're testing something useful. Also, while the PTS does some hardware profiling before each run (see representative output at http://openbenchmarking.org/result/1401071-UT-BUKOWSKIW54 ), it is insufficient to really do a scientific analysis of hardware's contributions to the scores. For example, there is no way to query openbenchmarking.org to see a graph of all the results for test X on systems with CPU Y and harddrive Z and RAM speed Q vs the amount of installed memory, with multiple results plotted as range bars. I would really like to be able to do that. In fact, the cross-platform nature of the PTS makes it harder to collect such information. Finally, the PTS doesn't have any ability to run tests on a cluster of machines. That is critical for testing any subsystem that involves networking, for example NFS. For these reasons, I set out to write my own framework. At a very high level, it provides a framework that handles common functionality like reporting results, commanding slave nodes, profiling the system hardware, etc. The individual benchmark programs are each written as ruby scripts that are executed by the framework. Importantly, the framework does not include any kind of built-in sequencer. There is no way to say "run all benchmarks". I envision that a technician would be responsible to selecting which benchmarks to run with which configuration options based on an organizations current needs. In a CI setting, there would be a short sh script that would run several benchmarks in series. In any case, the result report format does not assume anything about how the tests were sequenced. Each result enters the database as a separate record with full information about its configuration and the hardware and software environment under which it ran. Unfortunately, my framework is extremely incomplete. It's not even good enough for internal use, much less a wider audience. And I fear that my bosses won't give me any more time to work on it. It's also written in Ruby and uses STAF to command slave nodes, which the FreeBSD community might not be excited about. However, if there is any interest, I can ask for permission to share my design as a starting point for a more general framework. -Alan > > -- > Julio Merino / @jmmv > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 17:20:32 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 737D3FA9; Tue, 7 Jan 2014 17:20:32 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E8F041B3C; Tue, 7 Jan 2014 17:20:31 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id s07HKPUR008107; Tue, 7 Jan 2014 19:20:25 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s07HKPUR008107 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id s07HKPwb008106; Tue, 7 Jan 2014 19:20:25 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 7 Jan 2014 19:20:25 +0200 From: Konstantin Belousov To: Alexander Motin Subject: Re: UMA caches draining Message-ID: <20140107172025.GP59496@kib.kiev.ua> References: <52CB4F7D.2080909@FreeBSD.org> <20140107054825.GI59496@kib.kiev.ua> <52CBCC4F.8020900@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="PRJsLXMT40XvOKNo" Content-Disposition: inline In-Reply-To: <52CBCC4F.8020900@FreeBSD.org> User-Agent: Mutt/1.5.22 (2013-10-16) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 17:20:32 -0000 --PRJsLXMT40XvOKNo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jan 07, 2014 at 11:43:43AM +0200, Alexander Motin wrote: > On 07.01.2014 07:48, Konstantin Belousov wrote: > > On Tue, Jan 07, 2014 at 02:51:09AM +0200, Alexander Motin wrote: > >> I have some questions about memory allocation. At this moment our UMA > >> never returns freed memory back to the system until it is explicitly > >> asked by pageout daemon via uma_reclaim() call, that happens only when > >> system is quite low on memory. How does that coexist with buffer cache > >> and other consumers? Will, for example, buffer cache allocate buffers > >> and be functional when most of system's memory uselessly consumed by U= MA > >> caches? Is there some design how that supposed to work? > > Allocation of the pages which consitute a new buffer creates the pressu= re > > and causes pagedaemon wakeup if amount of free pages is too low. Look > > at the vm_page_grab() call in allocbuf(). Also note that buffer cache > > is not shrinked in response to the low memory events, and buffers pages > > are excluded from the page daemon scans since pages are wired. >=20 > Thanks. I indeed can't see respective vm_lowmem handler. But how does it= =20 > adapt then? It should have some sort of back pressure. And since it=20 > can't know about UMA internals, it should probably just see that system= =20 > is getting low on physical memory. Won't it shrink itself first in such= =20 > case before pagedaemon start its reclaimage? Buffer cache only caches buffers, it is not supposed to provide the file content cache, at least for VMIO. The buffer cache size is capped during system configuration, the algorithm to calculate the cache size is not easy to re-type, but look at the vfs_bio.c:kern_vfs_bio_buffer_alloc(). On the modern machines with 512MB of RAM of more, it is essentially 10% of the RAM which is dedicated to buffer cache. >=20 > When vm_lowmem condition finally fire, it will purge different data from= =20 > different subsystems, potentially still usable. UMA caches though have=20 > no valid data, only an allocation optimization. Shouldn't they be freed= =20 > first somehow, at least an unused part, as in my patch? Also I guess=20 > having more really free memory should make M_NOWAIT allocations to fail= =20 > less often. IMO this is not a right direction. My opinion is that M_NOWAIT allocation should be mosty banned from the top-level of the kernel, and then interrupt threads and i/o path should try hard to avoid allocations at all. Purging UMA caches on first sign of low memory condition would make UMA slower, possibly much slower for many workloads which are routinely handled now. Our code is accustomed to fast allocators, look at how many allocations typical syscall makes for temp buffers. Such change requires profiling of varying workloads to prove that it does not cause regressions. I suspect that what you do is tailored for single (ab)user of UMA. You might try to split UMA low memory handler into two, one for abuser, and one for the rest of caches. >=20 > >> I've made an experimental patch for UMA > >> (http://people.freebsd.org/~mav/drain_unused.patch) to make it every 20 > >> seconds return back to the system cached memory, unused for the last 20 > >> seconds. Algorithm is quite simple and patch seems like working, but I > >> am not sure whether I am approaching problem from the right side. Any > >> thoughts? >=20 > --=20 > Alexander Motin --PRJsLXMT40XvOKNo Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJSzDdYAAoJEJDCuSvBvK1B2jUP/3oSrPY1OWe65n/zdGXHTlu7 hw932AnxNCqLlUFsw8jAXp8x2tHf3mIZjCbVSFbz9Axjz5Q3G7cAlDLJJDGaY4qL FVZPi7VVU5/YODNkOAImIIZZtjjVNH4gGdLu22k+8mRSaA2/T2lLt0TdLf8aphIY iIGVkxaQOVymDQkoutHk3Jzn3gzAfc/c0gPY8ZwgIfzKi0YPFWHEvuB2SX4bdkes W846N53qKovjB6mNeIC2m20QL6vYWIzs/3B9BLRLNyvWpJIeDZx+WVtdWMN25Q/+ 934nFuZYGEItNafoTj90CsgXjWdwoQA9WO9QDxYhbiZ6TlhsPNjw7DAobY5OiOpJ TITx7k6bMrSkvrSAfVva3pBLK1aAmgxxaBTy+KXZcsCt658Afl3BsUJhv9J+IEyh sW8i2mDFsyq2cca1EiVmpTUpPDFZl9oc0kiN8Zlq5OET+NyeiPfVerauKSSmesSp 7+pWRPJae7veJvf2VAL8/zlt7MDaXHxYbqZpGoRsFW+xjXsOl9MDt8B0DdhVFO71 gE8KpRvJfbjRtRx3smGuDtlFb/QSUIsJo8bS7i2MtIFOeYto8JrAlJdVO7zpCadJ LcZWAtfXiM5xLbKkpof+9q95tE8w9VobtDvTJh7xNlACS1xOlICYS17nv0gCGMp0 YhLD5j3XgjN0lLUci2Zn =H1Fo -----END PGP SIGNATURE----- --PRJsLXMT40XvOKNo-- From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 17:49:45 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4D506EE; Tue, 7 Jan 2014 17:49:45 +0000 (UTC) Received: from mail-vb0-x22b.google.com (mail-vb0-x22b.google.com [IPv6:2607:f8b0:400c:c02::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6D4811E6C; Tue, 7 Jan 2014 17:49:45 +0000 (UTC) Received: by mail-vb0-f43.google.com with SMTP id p6so352080vbe.2 for ; Tue, 07 Jan 2014 09:49:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=Yq0XILyiZOwquI0eFrGMlTL65fbhSiqBzIVs23Qsv8g=; b=KYzLhF7RbywWTg9inpu2RQj9zJ54m8LvwVQ2zFfB5rOUoYpJNAF6icGbT2tfoh4pgU kWwfw19wN2JXK0HyKlx2eLfyqRg6nNKPNgnO1aw/e35jXxRGwvL5F0AE4OZMcBD47Yly 1U2TehblzBygmHUERhBQUM92qP0+G0BPmFo4GIxdm+/kovJtS/zuC5/ul9TOyfuvgALg QRZf44mNdyFSJjEZPitOTHdsufiEvdhM6CW8eF8VghHWAF4UCHyymSMMu/eJGi97fnw7 HO7p1DpUOwEH1sWu6vWf92fzZYzXJGaKMR9VKhsbzf0MwjaZarC1CM6hlsxTJyfyViA0 282A== X-Received: by 10.58.187.129 with SMTP id fs1mr4154454vec.45.1389116982905; Tue, 07 Jan 2014 09:49:42 -0800 (PST) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.58.134.98 with HTTP; Tue, 7 Jan 2014 09:49:02 -0800 (PST) In-Reply-To: References: From: Ivan Voras Date: Tue, 7 Jan 2014 18:49:02 +0100 X-Google-Sender-Auth: uHaQjcPuSeT4GJwFGBLBhs6tOCY Message-ID: Subject: Re: Continual benchmarking / regression testing? To: Alan Somers Content-Type: text/plain; charset=UTF-8 Cc: Julio Merino , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 17:49:45 -0000 On 7 January 2014 18:01, Alan Somers wrote: > Unfortunately, my framework is extremely incomplete. It's not even > good enough for internal use, much less a wider audience. And I fear > that my bosses won't give me any more time to work on it. It's also > written in Ruby and uses STAF to command slave nodes, which the > FreeBSD community might not be excited about. However, if there is > any interest, I can ask for permission to share my design as a > starting point for a more general framework. I don't think I have the time+spare brainpower to learn Ruby (I'm a Pythonista) but it would be interested to see the design you've created. I've looked at STAF before and I'm curious why you picked it? It seemed a combination of being antiquated and an overkill to me... From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 18:02:20 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02E37AB7 for ; Tue, 7 Jan 2014 18:02:20 +0000 (UTC) Received: from mail-we0-x230.google.com (mail-we0-x230.google.com [IPv6:2a00:1450:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8C4BD1F87 for ; Tue, 7 Jan 2014 18:02:19 +0000 (UTC) Received: by mail-we0-f176.google.com with SMTP id p61so464347wes.7 for ; Tue, 07 Jan 2014 10:02:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=date:from:to:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; bh=9f57Q8O0TZ+8qRB76xtv168V26rOWGGJG6QYIfdhUq0=; b=RiSGXp4aCv8HQcbaLqqYj6kOati/JSI/uJmpuWBRzg1ULt135JYL39tsLD4RkKiQiS 1ycZxeY3XawEFYLhBYuk0E3fPsZxtDnM6St77lkeJQY73AFjzabzb98XfQlA+i48uW4S xDhs/MQj9hVEUoBA9RQdeMmlQe61d76LDnp/lXRE1nrbt/9Gk3iVM4ZvvD/jyoFD+LW5 miNhAwamX+xLHXD1/1mwIEZUxdSYk2ZfnKfYPf2a5kkhgoODb3LFxbsUbiz7Le03Y3dv t2V7pGowEVjJbqIqKblE95viu0gKQQMDXfxzNlKdJ82MwvI33o52M9/uAgllagpXNwOS wrCg== X-Received: by 10.180.38.8 with SMTP id c8mr18054595wik.48.1389117737763; Tue, 07 Jan 2014 10:02:17 -0800 (PST) Received: from gumby.homeunix.com (87-194-105-247.bethere.co.uk. [87.194.105.247]) by mx.google.com with ESMTPSA id mt2sm4702696wic.7.2014.01.07.10.02.16 for (version=SSLv3 cipher=RC4-SHA bits=128/128); Tue, 07 Jan 2014 10:02:17 -0800 (PST) Date: Tue, 7 Jan 2014 18:02:16 +0000 From: RW To: freebsd-hackers@freebsd.org Subject: Re: FreeBSD-10 and processes swapping out for no reason Message-ID: <20140107180216.668b133d@gumby.homeunix.com> In-Reply-To: References: X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 18:02:20 -0000 On Tue, 7 Jan 2014 20:16:42 +0400 Dmitry Sivachenko wrote: > Hello! > ardware. > > I noticed that some of the processes on 10.0 boxes are marked as > swapped out in top(1) output: Do you have vm.swap_idle_enabled set? > I turned off swap: > # swapinfo > Device 1K-blocks Used Avail Capacity > # > > And state of these processes did not change: still second characted > in state field of ps output is W, and top shows them in angles. I've noticed that with swap_idle_enabled you can have processes shown as swapped, but zero swap in use. If it's a bug it's a long standing one. My guess is that the vm system does something to the pages so that they can be paged-out preferentially. From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 18:08:17 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B670FCE8 for ; Tue, 7 Jan 2014 18:08:17 +0000 (UTC) Received: from mail-lb0-f174.google.com (mail-lb0-f174.google.com [209.85.217.174]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3D7591FDA for ; Tue, 7 Jan 2014 18:08:16 +0000 (UTC) Received: by mail-lb0-f174.google.com with SMTP id y6so540984lbh.19 for ; Tue, 07 Jan 2014 10:08:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=0VFf1abSTpCb30sqddgD6xifZdF2M+gOcmcUByMBYZQ=; b=BwaOvPymutLYokqyOJlnLNe02aO+33+OXL5YiXMVbuaeHpcs/VVDL3JwvzeWbLYGqn rnil4O9odGkujYYBRTCaqYP0ywOXbyaLDChmeYSf8QlMdHoZMwauYZSzqM7XtpEvuFB7 OfuBAPu4dNA8rBlB0bX0194argSafC66ajK6kOGII0KiXOvEaaDXiIKwGE7Dik8JxpML RHNjyonEP46Ki1YeBOHAqfRh4NRS6RVyiGk2LrP74vuubIw7UZ3uaYRkzZRxezN0BuIG yEQz5k8jjjUPq34gsZh4qlPIUQwKMRH4M/vm/m14JjUMFOrnaRekn1LhZ+nRdNbbbFYc /KZw== X-Gm-Message-State: ALoCoQk+ow3xWj2h+IuYn8E0DMuPET+530Mr/eSqDSkjNw/g3SlDF9CJ++udtW/0gFptiISZ+6zz X-Received: by 10.152.4.162 with SMTP id l2mr3264124lal.75.1389118088651; Tue, 07 Jan 2014 10:08:08 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.129.101 with HTTP; Tue, 7 Jan 2014 10:07:48 -0800 (PST) X-Originating-IP: [2620:0:1040:406:9098:ad1a:255:209e] In-Reply-To: References: From: Julio Merino Date: Tue, 7 Jan 2014 18:07:48 +0000 Message-ID: Subject: Re: Continual benchmarking / regression testing? To: Ivan Voras Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 18:08:17 -0000 On Tue, Jan 7, 2014 at 4:17 PM, Ivan Voras wrote: > On 7 January 2014 17:11, Julio Merino wrote: >> On Tue, Jan 7, 2014 at 4:09 PM, Ivan Voras wrote: >>> Hello, >>> >>> Is someone working on a contitual benchmarking / regression testing >>> project for FreeBSD? I seem to recall there was a post several months >>> ago but I can't find it. >> >> See http://wiki.freebsd.org/TestSuite for the current efforts. > > Ok, by looking at the wiki page and http://kyua1.nyi.freebsd.org/ > (only stable/10 is available btw), it looks like this effort is > centered on correctness, not performance benchmarking? Correct. I have basically no thoughts at the moment on benchmarking, but that's certainly something worth tackling. My email crossed Alan's, but here go my thoughts anyway. I feel integrating some basic kind of performance testing into the test suite might be beneficial if only to catch regressions, but detailed performance testing may be difficult for all the reasons mentioned by Alan. A simple possibility could be to explicitly mark specific tests as "benchmark tests" so that Kyua could measure and record their run time. In fact, Kyua already records the run time of tests and maintains historical data. What would be missing is a way to graph the results and to alert when the measurements differ above some thresholds. But then, benchmarking tests will have special requirements -- particularly during the setup of dependencies, the setup of the machine (to ensure there is no background noise) and also due to the large amount of tunables that may be involved. Plugging such tests into a correctness test suite, except for simple tests, is hard and may be not such a great idea. > Is Kyua easily > adaptable to include graphs and other more visually attractive > presentation types? Not at the moment, but better reporting is the thing I want to tackle the soonest. See the planning details (http://julipedia.meroh.net/2014/01/freebsd-test-suite-goals-and-planning.html) for some more information. In particular, visit the "test matrix" sheet of the planning spreadsheet. But again, there is nothing there regarding performance testing. -- Julio Merino / @jmmv From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 18:15:42 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 251E8F7E for ; Tue, 7 Jan 2014 18:15:42 +0000 (UTC) Received: from mail-ea0-x22e.google.com (mail-ea0-x22e.google.com [IPv6:2a00:1450:4013:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A5B7310C5 for ; Tue, 7 Jan 2014 18:15:41 +0000 (UTC) Received: by mail-ea0-f174.google.com with SMTP id b10so371823eae.33 for ; Tue, 07 Jan 2014 10:15:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=+gaKGRyXWG4LjztInHt6SnzOIsW4sdOD2/yUBz3ifwg=; b=okI71P4KYM/178SZyo2hBvEL17Agauco0rB0CrFQwfKQriejVM8+zrCjjD7577kNtA 6o2kTKsDI3uxP0zAg11WvlcgERe6HiKN8kpnJ/KDWJmeO3JWnjQuqiSQXng4V3MoFIXb I5G9DX5BfVomoXpsrHimf60rR8zoz5khnBsa6ZpIiAChUwlGixJZ4BWOmWZBCK9sK+B+ zx6D1ME7iJVFS8LDz/T4oMPjRJfuDULOH3Q6Z/t4M5Z6a7nCXb+aHt8V7OP6orfkOmug 16gAgT0EHOmKJ3oUkDrcwSh8yRBN+f/5K+elvG/Ag4seDYSDQ0uQ/3mT50wHAnzkzCPH gd4g== X-Received: by 10.14.37.131 with SMTP id y3mr93679395eea.1.1389118540037; Tue, 07 Jan 2014 10:15:40 -0800 (PST) Received: from mavbook.mavhome.dp.ua ([134.249.139.101]) by mx.google.com with ESMTPSA id m1sm182151278eeg.0.2014.01.07.10.15.37 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 07 Jan 2014 10:15:39 -0800 (PST) Sender: Alexander Motin Message-ID: <52CC4448.90405@FreeBSD.org> Date: Tue, 07 Jan 2014 20:15:36 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: UMA caches draining References: <52CB4F7D.2080909@FreeBSD.org> <20140107054825.GI59496@kib.kiev.ua> <52CBCC4F.8020900@FreeBSD.org> <20140107172025.GP59496@kib.kiev.ua> In-Reply-To: <20140107172025.GP59496@kib.kiev.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 18:15:42 -0000 On 07.01.2014 19:20, Konstantin Belousov wrote: > On Tue, Jan 07, 2014 at 11:43:43AM +0200, Alexander Motin wrote: >> On 07.01.2014 07:48, Konstantin Belousov wrote: >>> On Tue, Jan 07, 2014 at 02:51:09AM +0200, Alexander Motin wrote: >>>> I have some questions about memory allocation. At this moment our UMA >>>> never returns freed memory back to the system until it is explicitly >>>> asked by pageout daemon via uma_reclaim() call, that happens only when >>>> system is quite low on memory. How does that coexist with buffer cache >>>> and other consumers? Will, for example, buffer cache allocate buffers >>>> and be functional when most of system's memory uselessly consumed by UMA >>>> caches? Is there some design how that supposed to work? >>> Allocation of the pages which consitute a new buffer creates the pressure >>> and causes pagedaemon wakeup if amount of free pages is too low. Look >>> at the vm_page_grab() call in allocbuf(). Also note that buffer cache >>> is not shrinked in response to the low memory events, and buffers pages >>> are excluded from the page daemon scans since pages are wired. >> >> Thanks. I indeed can't see respective vm_lowmem handler. But how does it >> adapt then? It should have some sort of back pressure. And since it >> can't know about UMA internals, it should probably just see that system >> is getting low on physical memory. Won't it shrink itself first in such >> case before pagedaemon start its reclaimage? > Buffer cache only caches buffers, it is not supposed to provide the file > content cache, at least for VMIO. The buffer cache size is capped during > system configuration, the algorithm to calculate the cache size is not > easy to re-type, but look at the vfs_bio.c:kern_vfs_bio_buffer_alloc(). > On the modern machines with 512MB of RAM of more, it is essentially 10% > of the RAM which is dedicated to buffer cache. So it is hard capped and never returns that memory in any case? 10% is not much, but it still doesn't sound perfect. >> When vm_lowmem condition finally fire, it will purge different data from >> different subsystems, potentially still usable. UMA caches though have >> no valid data, only an allocation optimization. Shouldn't they be freed >> first somehow, at least an unused part, as in my patch? Also I guess >> having more really free memory should make M_NOWAIT allocations to fail >> less often. > IMO this is not a right direction. My opinion is that M_NOWAIT > allocation should be mosty banned from the top-level of the kernel, and > then interrupt threads and i/o path should try hard to avoid allocations > at all. OK, M_NOWAIT possibly was bad example, though we have a lot of M_NOWAIT allocations in many important areas of the kernel. But still, making M_WAITOK allocation process to wait in case where system could already be prepared at hopefully low cost is possibly not perfect too. > Purging UMA caches on first sign of low memory condition would make UMA > slower, possibly much slower for many workloads which are routinely > handled now. Our code is accustomed to fast allocators, look at how many > allocations typical syscall makes for temp buffers. Such change > requires profiling of varying workloads to prove that it does not cause > regressions. Full purging on low memory is what present implementation actually does. I was proposing much softer alternative, purging only caches unused for last 20 seconds, that in some situations could allow to avoid full purges completely. > I suspect that what you do is tailored for single (ab)user of UMA. You > might try to split UMA low memory handler into two, one for abuser, and > one for the rest of caches. IMO the only "abuse" of ZFS is that it takes UMA and tries to use it for serious things, size of which is significant in total amount of RAM. And obviously it wants to do it fast too. But general problem of UMA is not new: with increasing number of zones, fluctuating load pattern will make different zones grow in different time, that at some point will inevitably create memory pressure, even if each consumer or even all of them together are size-capped. ZFS just brings that up to the limit, actively using up to 90 different zones. But the problem is not new. If you prefer to see UMA consumers divided into some classes -- fine (though IMO that is very non-obvious how to decide that in every case), but what would you see logic there? Should there be another memory limit, like low and high watermarks? Aren't there any benefits of freeing RAM preventively, when there still "enough" free? Shouldn't we mix "soft" and "hard" purges at some rate between low and high watermarks to keep all consumers feeling fair amount of pressure, depending on their class? >>>> I've made an experimental patch for UMA >>>> (http://people.freebsd.org/~mav/drain_unused.patch) to make it every 20 >>>> seconds return back to the system cached memory, unused for the last 20 >>>> seconds. Algorithm is quite simple and patch seems like working, but I >>>> am not sure whether I am approaching problem from the right side. Any >>>> thoughts? -- Alexander Motin From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 19:33:28 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3D8023D9; Tue, 7 Jan 2014 19:33:28 +0000 (UTC) Received: from mail-wg0-x231.google.com (mail-wg0-x231.google.com [IPv6:2a00:1450:400c:c00::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A45671653; Tue, 7 Jan 2014 19:33:27 +0000 (UTC) Received: by mail-wg0-f49.google.com with SMTP id x12so566854wgg.4 for ; Tue, 07 Jan 2014 11:33:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=E8jtLqdjkfn2mU4zDqNdg0qZp1+KkBmpsupiNvJCAfQ=; b=0pOHZkxBWqSH8wj4zruHXu5zDzUV9Vey1Elz5WWhLaUs0mCIAlh5c+xmlf/Iq+cSaG px6XZP/HPClWovRhg/XP9RFf5jJtQ3q/BJineUEXtBxA8A2TqSeSbaDAKXZ1hgVKp4Ug yubkQjNW3+UfaMPEvuS54/pnbCWl9bw9npjlRqTW01CU1ECOc9qHhfYX2rLgOP5gU2fe cFR4GHLnahqFdGgdSpSiQvC1fC7d9tR2Jflw5u3ImE8DfVDQa6vEOJf9X/iW2ThnClR0 gIbZsmOghFBZw/5njUeLiVm/jDqA7k7XXPP+/0REIR2rToknBoDTkxlsBjq2Zg3PPMh5 te0w== MIME-Version: 1.0 X-Received: by 10.180.37.69 with SMTP id w5mr18116617wij.53.1389123206030; Tue, 07 Jan 2014 11:33:26 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Tue, 7 Jan 2014 11:33:25 -0800 (PST) In-Reply-To: References: Date: Tue, 7 Jan 2014 12:33:25 -0700 X-Google-Sender-Auth: N4Pw2HhpsA3KVCsyTthw5Xw59e8 Message-ID: Subject: Re: Continual benchmarking / regression testing? From: Alan Somers To: Ivan Voras Content-Type: text/plain; charset=ISO-8859-1 Cc: Julio Merino , "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 19:33:28 -0000 On Tue, Jan 7, 2014 at 10:49 AM, Ivan Voras wrote: > On 7 January 2014 18:01, Alan Somers wrote: >> Unfortunately, my framework is extremely incomplete. It's not even >> good enough for internal use, much less a wider audience. And I fear >> that my bosses won't give me any more time to work on it. It's also >> written in Ruby and uses STAF to command slave nodes, which the >> FreeBSD community might not be excited about. However, if there is >> any interest, I can ask for permission to share my design as a >> starting point for a more general framework. > > I don't think I have the time+spare brainpower to learn Ruby (I'm a > Pythonista) but it would be interested to see the design you've > created. > > I've looked at STAF before and I'm curious why you picked it? It > seemed a combination of being antiquated and an overkill to me... I picked Ruby and STAF for the same reason: we're already using them at $JOB. In the case of STAF, we were already using it as part of our continuous install and test system. We aren't using very many of its capabilities, though. In particular, we've long since switched from STAX to ATF/Kyua. Basically, we only use STAF to execute commands on a remote machine. It offers a little more control than using SSH. We also use the STAF semaphore service to ensure exclusive access to both test machines and network slaves. From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 19:41:31 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3EB2B777; Tue, 7 Jan 2014 19:41:31 +0000 (UTC) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A68B216F9; Tue, 7 Jan 2014 19:41:30 +0000 (UTC) Received: by mail-wi0-f173.google.com with SMTP id hn9so4616885wib.6 for ; Tue, 07 Jan 2014 11:41:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=nHpq0mTCEL+Mq4YHYVGtQR0MFTs8aXQsHexl777Ruu0=; b=bG3KWIPeQPZbey2x4c7EkW3LP2TwGOXBIvAf7PH+VQuNmtDYvl/QrlBkf+m1PYFR6z 1NFqJvc7GFVXEy9XLFJzg5AEKS1bOk5+0o+CCO26QUyDCOLpOFOXz5zqo6SBnik0NPQb okTJknCpLg1xjaqYGPKpupiKgkgK4VS76MofJLb6/Ubjc6bE/e+GZTkcEfmsMAkNIv+y gSbiP2tH08SOvTfVpzwDfTRAgacQy7RMwtFhhElLWMCmSQJA5tblqd/X8UeiRPErhwHC KH0EkjMwEgXWNiq5ahvrRCUBCS4In3Zzyrdcv6eEi3sfdaTsQs27q0dTKYC0AdHxRnNu +qnw== MIME-Version: 1.0 X-Received: by 10.180.37.69 with SMTP id w5mr18142482wij.53.1389123689184; Tue, 07 Jan 2014 11:41:29 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Tue, 7 Jan 2014 11:41:29 -0800 (PST) In-Reply-To: References: Date: Tue, 7 Jan 2014 12:41:29 -0700 X-Google-Sender-Auth: Cn6Up_6e3aKYIjXPj5LCSB6421E Message-ID: Subject: Re: Continual benchmarking / regression testing? From: Alan Somers To: Julio Merino Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers , Ivan Voras X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 19:41:31 -0000 On Tue, Jan 7, 2014 at 11:07 AM, Julio Merino wrote: > On Tue, Jan 7, 2014 at 4:17 PM, Ivan Voras wrote: >> On 7 January 2014 17:11, Julio Merino wrote: >>> On Tue, Jan 7, 2014 at 4:09 PM, Ivan Voras wrote: >>>> Hello, >>>> >>>> Is someone working on a contitual benchmarking / regression testing >>>> project for FreeBSD? I seem to recall there was a post several months >>>> ago but I can't find it. >>> >>> See http://wiki.freebsd.org/TestSuite for the current efforts. >> >> Ok, by looking at the wiki page and http://kyua1.nyi.freebsd.org/ >> (only stable/10 is available btw), it looks like this effort is >> centered on correctness, not performance benchmarking? > > Correct. I have basically no thoughts at the moment on benchmarking, > but that's certainly something worth tackling. > > My email crossed Alan's, but here go my thoughts anyway. I feel > integrating some basic kind of performance testing into the test suite > might be beneficial if only to catch regressions, but detailed > performance testing may be difficult for all the reasons mentioned by > Alan. For a good example of performance regression testing, see the Linux kernel tracker at http://www.phoromatic.com/kernel-tracker.php? . As you can see, the hardware and software profiling is weak, and regression detection is very inexact. It wouldn't be possible to distill a single benchmark's results into a "regressed"/"didn't regress" result. Accurately detecting regressions would probably take some moderately sophisticated statistics. > > A simple possibility could be to explicitly mark specific tests as > "benchmark tests" so that Kyua could measure and record their run > time. In fact, Kyua already records the run time of tests and > maintains historical data. What would be missing is a way to graph > the results and to alert when the measurements differ above some > thresholds. > > But then, benchmarking tests will have special requirements -- > particularly during the setup of dependencies, the setup of the > machine (to ensure there is no background noise) and also due to the > large amount of tunables that may be involved. Plugging such tests > into a correctness test suite, except for simple tests, is hard and > may be not such a great idea. Yeah, that's basically the same conclusion I came to. > >> Is Kyua easily >> adaptable to include graphs and other more visually attractive >> presentation types? > > Not at the moment, but better reporting is the thing I want to tackle > the soonest. See the planning details > (http://julipedia.meroh.net/2014/01/freebsd-test-suite-goals-and-planning.html) > for some more information. In particular, visit the "test matrix" > sheet of the planning spreadsheet. But again, there is nothing there > regarding performance testing. So is Julipedia now the best source for Kyua news instead of http://engineering-kyua.blogspot.com/ ? I have the latter in my RSS feed, but it hasn't updated for awhile. -Alan > > -- > Julio Merino / @jmmv > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 7 18:55:08 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EAD7AAFF for ; Tue, 7 Jan 2014 18:55:08 +0000 (UTC) Received: from mail-lb0-x236.google.com (mail-lb0-x236.google.com [IPv6:2a00:1450:4010:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 712C4137B for ; Tue, 7 Jan 2014 18:55:08 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id l4so588233lbv.27 for ; Tue, 07 Jan 2014 10:55:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=401r8ph6mCUaitqlLZzLivswL1zfFhP0qIkVN/Ho6B0=; b=I5LS7DtBiYXBHS7v6A7t/bk4PyPWI/hAcduJt++Q9tLk82auzC9o5KVMp8gDRM3eQU srUhaHjbZIVoJtPq9eC8G+naBK/wdMWlNcAAgmKecWWFWpmEuLTqO5rzmmHj1bmpySZW V+C+yeb+gsY1fnoUFifloWuHiJGHT1iVLdvaUi/mIlgy+JkUf0OM3sjpIucM0JBcEa3g XOAtciEj7itKF4hN/BAnQWC7Rzn+n4qm2U1uFi2N1/9l7i9pWVJRqUE9BErQu4fRFNY+ WCGsjLr+3a7SicpJk+Uhg5OtSXIpmPjQWas3lkPVdOQ14I7E4e6uKViR4S6DlY6AZZzI 9j1Q== X-Received: by 10.112.168.199 with SMTP id zy7mr3198272lbb.68.1389120906119; Tue, 07 Jan 2014 10:55:06 -0800 (PST) Received: from [10.0.1.20] ([176.193.50.79]) by mx.google.com with ESMTPSA id di11sm58364788lac.0.2014.01.07.10.55.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 07 Jan 2014 10:55:05 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: FreeBSD-10 and processes swapping out for no reason From: Dmitry Sivachenko In-Reply-To: Date: Tue, 7 Jan 2014 22:55:03 +0400 Content-Transfer-Encoding: quoted-printable Message-Id: <5AC060E3-072E-41E6-94FE-372A94E3084E@gmail.com> References: To: hackers@freebsd.org, rwmaillists@googlemail.com X-Mailer: Apple Mail (2.1827) X-Mailman-Approved-At: Wed, 08 Jan 2014 05:51:58 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Jan 2014 18:55:09 -0000 On 07 =D1=8F=D0=BD=D0=B2. 2014 =D0=B3., at 20:16, Dmitry Sivachenko = wrote: > >Hello! >=20 > >Recently I updated several of my servers from 9.2-STABLE 10.0: > >10.0-PRERELEASE FreeBSD 10.0-PRERELEASE #0 r259661 > > > >Workload did not change at all, same programs, same load, same = hardware. > > > >I noticed that some of the processes on 10.0 boxes are marked as = swapped out in top(1) output: > > > >1436 root 1 43 0 16524K 0K nanslp 14 1:14 = 0.00% > >1381 smmsp 1 20 0 23988K 0K pause 18 0:04 = 0.00% >99348 mitya 1 21 0 23492K 0K pause 16 0:00 = 0.00% >=20 >Do you have vm.swap_idle_enabled set? No, it is set to zero (default).= From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 07:45:35 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 84EE0472; Wed, 8 Jan 2014 07:45:35 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 263B119B0; Wed, 8 Jan 2014 07:45:34 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id s087jOeU090851; Wed, 8 Jan 2014 09:45:24 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s087jOeU090851 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id s087jOHr090850; Wed, 8 Jan 2014 09:45:24 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 8 Jan 2014 09:45:24 +0200 From: Konstantin Belousov To: Alexander Motin Subject: Re: UMA caches draining Message-ID: <20140108074524.GT59496@kib.kiev.ua> References: <52CB4F7D.2080909@FreeBSD.org> <20140107054825.GI59496@kib.kiev.ua> <52CBCC4F.8020900@FreeBSD.org> <20140107172025.GP59496@kib.kiev.ua> <52CC4448.90405@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="RL2ThwiQkKBVVxrm" Content-Disposition: inline In-Reply-To: <52CC4448.90405@FreeBSD.org> User-Agent: Mutt/1.5.22 (2013-10-16) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 07:45:35 -0000 --RL2ThwiQkKBVVxrm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jan 07, 2014 at 08:15:36PM +0200, Alexander Motin wrote: > On 07.01.2014 19:20, Konstantin Belousov wrote: > > On Tue, Jan 07, 2014 at 11:43:43AM +0200, Alexander Motin wrote: > >> On 07.01.2014 07:48, Konstantin Belousov wrote: > >>> On Tue, Jan 07, 2014 at 02:51:09AM +0200, Alexander Motin wrote: > >>>> I have some questions about memory allocation. At this moment our UMA > >>>> never returns freed memory back to the system until it is explicitly > >>>> asked by pageout daemon via uma_reclaim() call, that happens only wh= en > >>>> system is quite low on memory. How does that coexist with buffer cac= he > >>>> and other consumers? Will, for example, buffer cache allocate buffers > >>>> and be functional when most of system's memory uselessly consumed by= UMA > >>>> caches? Is there some design how that supposed to work? > >>> Allocation of the pages which consitute a new buffer creates the pres= sure > >>> and causes pagedaemon wakeup if amount of free pages is too low. Look > >>> at the vm_page_grab() call in allocbuf(). Also note that buffer cache > >>> is not shrinked in response to the low memory events, and buffers pag= es > >>> are excluded from the page daemon scans since pages are wired. > >> > >> Thanks. I indeed can't see respective vm_lowmem handler. But how does = it > >> adapt then? It should have some sort of back pressure. And since it > >> can't know about UMA internals, it should probably just see that system > >> is getting low on physical memory. Won't it shrink itself first in such > >> case before pagedaemon start its reclaimage? > > Buffer cache only caches buffers, it is not supposed to provide the file > > content cache, at least for VMIO. The buffer cache size is capped during > > system configuration, the algorithm to calculate the cache size is not > > easy to re-type, but look at the vfs_bio.c:kern_vfs_bio_buffer_alloc(). > > On the modern machines with 512MB of RAM of more, it is essentially 10% > > of the RAM which is dedicated to buffer cache. >=20 > So it is hard capped and never returns that memory in any case? 10% is=20 > not much, but it still doesn't sound perfect. No, this is not how buffer cache works. VMIO buffers do not own pages, the memory is charged to the corresponding vnode vm_object. The reason why buffer cache page count is capped is due to buffer wiring the consituing pages. Before unmapped i/o, pages must be mapped into KVA, so the buffer cache wiring was naturally limited by the buffer map. After the unmapped, the buffer map is used only for mapping of metadata buffers, and the cap on the buffer cache limits amount of wired pages used by buffers, without the pressure from KVA. Buffers are recycled by getnewbuf() as usual, while vnode pages are managed by pagedaemon. So it is incorrect to state that buffers allocate memory for VMIO. >=20 > >> When vm_lowmem condition finally fire, it will purge different data fr= om > >> different subsystems, potentially still usable. UMA caches though have > >> no valid data, only an allocation optimization. Shouldn't they be freed > >> first somehow, at least an unused part, as in my patch? Also I guess > >> having more really free memory should make M_NOWAIT allocations to fail > >> less often. > > IMO this is not a right direction. My opinion is that M_NOWAIT > > allocation should be mosty banned from the top-level of the kernel, and > > then interrupt threads and i/o path should try hard to avoid allocations > > at all. >=20 > OK, M_NOWAIT possibly was bad example, though we have a lot of M_NOWAIT= =20 > allocations in many important areas of the kernel. But still, making=20 > M_WAITOK allocation process to wait in case where system could already=20 > be prepared at hopefully low cost is possibly not perfect too. >=20 > > Purging UMA caches on first sign of low memory condition would make UMA > > slower, possibly much slower for many workloads which are routinely > > handled now. Our code is accustomed to fast allocators, look at how many > > allocations typical syscall makes for temp buffers. Such change > > requires profiling of varying workloads to prove that it does not cause > > regressions. >=20 > Full purging on low memory is what present implementation actually does.= =20 > I was proposing much softer alternative, purging only caches unused for= =20 > last 20 seconds, that in some situations could allow to avoid full=20 > purges completely. The vm_pageout_grow_cache() is called for the kmem_alloc() failures, which means that normal pageout of paged memory fails to produce enough fresh free pages. In particular, it must only happen when there is significant misaliance between user/pageable and kernel/unpageable allocations. Purging all kernel caches sounds reasonable than, while purging caches once in 20 seconds is useless if kmem_alloc() requests are satisfied by pagedaemon. >=20 > > I suspect that what you do is tailored for single (ab)user of UMA. You > > might try to split UMA low memory handler into two, one for abuser, and > > one for the rest of caches. >=20 > IMO the only "abuse" of ZFS is that it takes UMA and tries to use it for= =20 > serious things, size of which is significant in total amount of RAM. And= =20 > obviously it wants to do it fast too. But general problem of UMA is not= =20 > new: with increasing number of zones, fluctuating load pattern will make= =20 > different zones grow in different time, that at some point will=20 > inevitably create memory pressure, even if each consumer or even all of= =20 > them together are size-capped. ZFS just brings that up to the limit,=20 > actively using up to 90 different zones. But the problem is not new. >=20 > If you prefer to see UMA consumers divided into some classes -- fine=20 > (though IMO that is very non-obvious how to decide that in every case),= =20 IMO the split is obvious, ZFS would mark it zones for frequent purge, while other kernel consumers would not. > but what would you see logic there? Should there be another memory=20 > limit, like low and high watermarks? Aren't there any benefits of=20 > freeing RAM preventively, when there still "enough" free? Shouldn't we=20 > mix "soft" and "hard" purges at some rate between low and high=20 > watermarks to keep all consumers feeling fair amount of pressure,=20 > depending on their class? >=20 > >>>> I've made an experimental patch for UMA > >>>> (http://people.freebsd.org/~mav/drain_unused.patch) to make it every= 20 > >>>> seconds return back to the system cached memory, unused for the last= 20 > >>>> seconds. Algorithm is quite simple and patch seems like working, but= I > >>>> am not sure whether I am approaching problem from the right side. Any > >>>> thoughts? >=20 > --=20 > Alexander Motin --RL2ThwiQkKBVVxrm Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJSzQITAAoJEJDCuSvBvK1BI7wP/1S4jYj1Rtlay71+tDB74NGW tlZKRyvlUFCZDwNxWVMuXNBwXNlRVTl5Lqkw8cGpNqGppxMHbN8vNdMQJC96cddN javrJPh4u721AYUo+m32v7Z1JM821Q4ZqPcDgtoYSarvXnToQy141TGyhts4npBB WWgrihIMAWwEaoEFkzWdKASdvt2N3xYyJPy4aYfsXJ3ALMv2AYRHVohh66j6DXKn jd29wfu9ZqtS8iE5PquXPsflPnD5dZtS5xnFFmWSP/cotA1kwpifXsJrNEXrNweM DAeNtTg++PBfVZXBpJCF4Cb3n03bmkdyi83GYiQyYi/tp5H+a6jLGKMEQJGCPnDW ug8d33/y6o1o/Auc2OpLFf4ARpLE+KUfmaQq4Hw6GhMLnpw2Kx7GMKqz2F7bZ0do KehoBO9/y1LwQVYpLesiYjBBMyjCmrq3IxEIu3ucj4rvJq/fehqWrk2VjUEWg5KH 2WE02qOlgwi4AiJDVz4VjCj2YMPb1QYSLkhrSc1X9+qnNnncVlYy07QEYN5n696e hRThLPNSiGc/JVWrzQnr5BrJaHX6CxGS3TIyMKvSIZ21JTWZi8tMl4XEmezE2oF0 stZZ78DbGGbB+WGtcVl1M+hmEcbGuC6rW+FgOi+HNxttUjqtfZDpmQ9cpCOvPu1t WKJ5rU0BrBIJJcvcgbA3 =3tkN -----END PGP SIGNATURE----- --RL2ThwiQkKBVVxrm-- From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 08:03:47 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D0FAA6D for ; Wed, 8 Jan 2014 08:03:47 +0000 (UTC) Received: from mail-qe0-x231.google.com (mail-qe0-x231.google.com [IPv6:2607:f8b0:400d:c02::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CF95E1B2A for ; Wed, 8 Jan 2014 08:03:46 +0000 (UTC) Received: by mail-qe0-f49.google.com with SMTP id w7so1447115qeb.36 for ; Wed, 08 Jan 2014 00:03:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=Oe4UIG0vyoFNisuPyKdQxkBUP/Wf934AmwYb2Nu8wS0=; b=NQj60L8j5lXLpa3ckftNVOCmCRUfH49P9afGEgZt64AYVMwgYgbyVKTRF8AXrtZtDz Y5x7y5/7tXoPon1ChGpLh2bKu+8KGdRIyecASMxdVe1efon03rODfOj32nf4LdiSSrnK ewc8TPv6WOHRIZomScCq1BT5tev2iO2F2EYiXdHScB/rXDy4dSztOlimDqK2y4PHLgB8 4D6HBWfOu5NZOoNQp1MEqRveIz1vx9hVOIFpxnEmsslu7DX8tB746MOcjUif2sbck7gv 2iz2JKAWvjisd3XWYSSpyzaHIFHcA4hd/oDcVkwbf1uRblePQinxgvVtm+yZvYPO5uP8 Z+Yw== MIME-Version: 1.0 X-Received: by 10.49.30.197 with SMTP id u5mr11355603qeh.33.1389168226110; Wed, 08 Jan 2014 00:03:46 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.52.8 with HTTP; Wed, 8 Jan 2014 00:03:46 -0800 (PST) In-Reply-To: References: Date: Wed, 8 Jan 2014 00:03:46 -0800 X-Google-Sender-Auth: sr1kLfpXW3KD12WaCki0OUHG2XQ Message-ID: Subject: Re: Working on NUMA support From: Adrian Chadd To: Andrew Bates Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 08:03:47 -0000 Cool! Do you have any working code to implement the API, or is this just in the design phase right now? -a On 6 January 2014 12:11, Andrew Bates wrote: > Hey all, > > My name is Andrew Bates, and I would like to take a bit of your time to > talk about NUMA support. > > Supporting Non-Uniform Memory Access in FreeBSD is something that has bee= n > brought up in the past < > http://freebsd.1045724.n5.nabble.com/NUMA-Support-is-there-in-FreeBSD-td4= 865200.html>. > This is becoming increasingly important now that multiprocessor > systems > are an expanding technology, thus performance is scaling in terms of cpu > count, rather than just clock rate. > > There is a great opportunity here to optimize performance. After being > asked to look into this by the EMC Isilon Storage Division, myself and a > few colleagues advised by Andrew Pilloud and Jeff Roberson would like to > propose APIs to handle basic memory allocation/management to specific NUM= A > domains. > > What we have devised so far consists of two levels. First there are the > KPIs, to expose NUMA functionality at a thread level of domain affinity. > Secondly, there would be a userspace/interface to take advantage of the > proposed APIs, thus giving users the capability to make their application= s > NUMA-aware. > > We took the time to look into how many other systems (Linux, Macintosh, > Solaris, Windows) already approach this problem, so there are some aspect= s > of our solution that are similar to how Linux and Solaris handle NUMA. > Unlike Linux libnuma, we are only proposing a few additions and a minimal > library that can easily be expanded later to suit users=92 needs. > > > KISS in mind, we came up with the following KPI prototypes (freebsdnuma.h= ) > to uncover NUMA in a usable fashion: > > > - > > cpuset_get_memory_affinity() > - > > cpuset_set_memory_affinity() > - > > move_pages() > - > > migrate_pages() > - > > get_numa_cpus() > - > > get_numa_weights() > > > Then to the second part, we have the following userspace API prototypes > (numanor.h) for our interface and testing purposes: > > > - > > is_numa_available() > - > > set_thread_on_domain() > - > > set_memory_policy() > - > > move_thread() > > > In much much more detail, you can learn more about these prototypes, this > project, view our progress, track along, and give input on our github rep= o > < https://github.com/andrewbates09/freebsd-numa > or simply via email. Th= is > repo currently includes fully commented prototypes (like a mini man page) > and will later include additions to the project. > > If anyone has any comments, suggestions, concerns, quandaries, or just > general thoughts please feel free to contact us, as we would love to hear > your input! > > The Leaders: Sakire Arslan Ay, Andrew Pilloud, Jeff Roberson > The Team: Andrew Bates, Joshua Clark, Alex Schuldberg, Dustin Walker > > -- > V/Respectfully, > Andrew M Bates > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 08:23:48 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6B9CB560; Wed, 8 Jan 2014 08:23:48 +0000 (UTC) Received: from mail-oa0-x22c.google.com (mail-oa0-x22c.google.com [IPv6:2607:f8b0:4003:c02::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 23F241DFA; Wed, 8 Jan 2014 08:23:48 +0000 (UTC) Received: by mail-oa0-f44.google.com with SMTP id h16so1448336oag.3 for ; Wed, 08 Jan 2014 00:23:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=7KbaJBub0PEUq+IuqydLIC1GIc3yqzFffAi+nbHvnwM=; b=ypKLq0adFDcP9ZSSzwpFNNM1+pFlxFl0obkNH7GE4gWqJUwLe2W12b4vlcvt8j5Ch9 Ug7a1e6iVg679NKeKqLkHc0FhVD8X/pYJUgieyA+bFwDERGxq0VIHW9+JbRGFtdbTS6U A8TgJJjllzc4iUO2arkEK4VSKcs4YhzqkkR+c0XKwHX0T7zrdu197jVT0oju274uHeee Q+Nk89gjkfE2NJOAhl9U54YfTslL+0nZAgFGYocTBMutMuxsOwIxSi5IpCDJyuBsNSJq DGx7ux6ko1IM8ktPW5RvtRLee1u63w50t0WpneGbl+0cidE51xiD93Uqwiv8HPdqk23f vksg== MIME-Version: 1.0 X-Received: by 10.182.232.4 with SMTP id tk4mr81799267obc.9.1389169427317; Wed, 08 Jan 2014 00:23:47 -0800 (PST) Received: by 10.182.166.39 with HTTP; Wed, 8 Jan 2014 00:23:47 -0800 (PST) In-Reply-To: References: Date: Wed, 8 Jan 2014 00:23:47 -0800 Message-ID: Subject: Re: Working on NUMA support From: Andrew Bates To: Adrian Chadd Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 08:23:48 -0000 Hey Adrian, We spent the last few months on research/design and plan on spending the next few months preparing tests, building the physical server(s) to test on, and finishing the prototype'd functions. There is some white-board and pen/paper design that hasn't yet made it to the github repo, but the prototypes online are hopefully a solid base for what we intend to complete. In the meantime, we are excited to hear who else is interested in NUMA and if anyone has suggestions or concerns about our approach. On Wed, Jan 8, 2014 at 12:03 AM, Adrian Chadd wrote: > Cool! Do you have any working code to implement the API, or is this > just in the design phase right now? > > > -a > > > On 6 January 2014 12:11, Andrew Bates wrote: > > Hey all, > > > > My name is Andrew Bates, and I would like to take a bit of your time to > > talk about NUMA support. > > > > Supporting Non-Uniform Memory Access in FreeBSD is something that has > been > > brought up in the past < > > > http://freebsd.1045724.n5.nabble.com/NUMA-Support-is-there-in-FreeBSD-td4= 865200.html > >. > > This is becoming increasingly important now that multiprocessor > > systems > > are an expanding technology, thus performance is scaling in terms of cp= u > > count, rather than just clock rate. > > > > There is a great opportunity here to optimize performance. After being > > asked to look into this by the EMC Isilon Storage Division, myself and = a > > few colleagues advised by Andrew Pilloud and Jeff Roberson would like t= o > > propose APIs to handle basic memory allocation/management to specific > NUMA > > domains. > > > > What we have devised so far consists of two levels. First there are th= e > > KPIs, to expose NUMA functionality at a thread level of domain affinity= . > > Secondly, there would be a userspace/interface to take advantage of th= e > > proposed APIs, thus giving users the capability to make their > applications > > NUMA-aware. > > > > We took the time to look into how many other systems (Linux, Macintosh, > > Solaris, Windows) already approach this problem, so there are some > aspects > > of our solution that are similar to how Linux and Solaris handle NUMA. > > Unlike Linux libnuma, we are only proposing a few additions and a minim= al > > library that can easily be expanded later to suit users=92 needs. > > > > > > KISS in mind, we came up with the following KPI prototypes > (freebsdnuma.h) > > to uncover NUMA in a usable fashion: > > > > > > - > > > > cpuset_get_memory_affinity() > > - > > > > cpuset_set_memory_affinity() > > - > > > > move_pages() > > - > > > > migrate_pages() > > - > > > > get_numa_cpus() > > - > > > > get_numa_weights() > > > > > > Then to the second part, we have the following userspace API prototypes > > (numanor.h) for our interface and testing purposes: > > > > > > - > > > > is_numa_available() > > - > > > > set_thread_on_domain() > > - > > > > set_memory_policy() > > - > > > > move_thread() > > > > > > In much much more detail, you can learn more about these prototypes, th= is > > project, view our progress, track along, and give input on our github > repo > > < https://github.com/andrewbates09/freebsd-numa > or simply via email. > This > > repo currently includes fully commented prototypes (like a mini man pag= e) > > and will later include additions to the project. > > > > If anyone has any comments, suggestions, concerns, quandaries, or just > > general thoughts please feel free to contact us, as we would love to he= ar > > your input! > > > > The Leaders: Sakire Arslan Ay, Andrew Pilloud, Jeff Roberson > > The Team: Andrew Bates, Joshua Clark, Alex Schuldberg, Dustin Walker > > > > -- > > V/Respectfully, > > Andrew M Bates > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > --=20 V/Respectfully, Andrew M Bates From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 08:35:42 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 61E4FB98 for ; Wed, 8 Jan 2014 08:35:42 +0000 (UTC) Received: from csmtp2.one.com (csmtp2.one.com [91.198.169.22]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 20BCC1EED for ; Wed, 8 Jan 2014 08:35:41 +0000 (UTC) Received: from [172.20.10.3] (94.191.184.148.mobile.3.dk [94.191.184.148]) by csmtp2.one.com (Postfix) with ESMTPA id 7C52410BA3 for ; Wed, 8 Jan 2014 08:28:47 +0000 (UTC) Received: from [172.20.10.3] (94.191.184.148.mobile.3.dk [94.191.184.148]) (using TLSv1 with cipher AES128-SHA) by 0.0.0.0:2500 (trex/4.8.87); Wed, 08 Jan 2014 08:28:48 GMT Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Continual benchmarking / regression testing? From: Erik Cederstrand In-Reply-To: Date: Wed, 8 Jan 2014 09:28:44 +0100 Cc: freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: X-Mailer: Apple Mail (2.1827) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 08:35:42 -0000 Den 07/01/2014 kl. 17.09 skrev Ivan Voras : > Hello, >=20 > Is someone working on a contitual benchmarking / regression testing > project for FreeBSD? I seem to recall there was a post several months > ago but I can't find it. I have an old project lying with a build/PXE server, a client installer = and benchmark runner and a website presenting benchmark reports. It=92s = a bit outdated, but maybe it can be a starting point? It=92s a mix of = shell and Python. See https://github.com/ecederstrand/perfmon. I=92d like to work with you on this, if you want. Erik= From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 09:18:16 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F0B148C5 for ; Wed, 8 Jan 2014 09:18:15 +0000 (UTC) Received: from nm34-vm2.bullet.mail.bf1.yahoo.com (nm34-vm2.bullet.mail.bf1.yahoo.com [72.30.239.74]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9305A1288 for ; Wed, 8 Jan 2014 09:18:15 +0000 (UTC) Received: from [66.196.81.172] by nm34.bullet.mail.bf1.yahoo.com with NNFMP; 08 Jan 2014 09:14:41 -0000 Received: from [98.139.213.10] by tm18.bullet.mail.bf1.yahoo.com with NNFMP; 08 Jan 2014 09:14:41 -0000 Received: from [127.0.0.1] by smtp110.mail.bf1.yahoo.com with NNFMP; 08 Jan 2014 09:14:41 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1389172481; bh=fvrNRgb9UnV0tLRdMG0AVzOeJpHjPDhEq4a+KTkmMoI=; h=X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:References:In-Reply-To:Mime-Version:Content-Transfer-Encoding:Content-Type:Message-Id:Cc:X-Mailer:From:Subject:Date:To; b=vSwby/yHylNns+sLlaicQuwfj9VEsRhmdhcTBDTwq6GsQozHn2x1+p5WBK0NWpbcTrr2lrIZ15/nHj+fNBAJsi3Eo95359XgKMmxC4YgvAGWX/30xP6Zj5L0Zv814dIc9u0azmtoPmR/+5b1yJGedIPGIT7jwSZHAd3evcxhThY= X-Yahoo-Newman-Id: 120882.31637.bm@smtp110.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: 27BLsVEVM1mSZhzXX1uusupsv.NjLAUj4DyolK02k6joRHy wE3Z1A8OSl0Fu6qdNF2uLqvHU8qcvZusBEWcYWib76g1cwbg3TqeydO8SfwR dGIv50B_PKrWGT2Gux5r45nWQRGpKLorphrAqP3JbI65qtxmTA48TGxTNZk0 1HqupN50JOQ7xHZym6s_SSLjlHrlRPd.aJAO.43W73es2xUR7Eg6YnAdpxhC 2a0KUuZkP9q1rCkKbHGHYRXlggPLYQjvLEclvdhve.3mPjWVLMcRbIALjNGi SwF2aMpvzyPlTUnoc6ZYiMiUKhDZGJ0auMSH5UTv9MFfpsAH8Ry9u9KI6eu9 eCYllxn9jMBy3_tQuHYh1n9XcbkcMIHq96EDY4zHOMrUtHupExUJaqFNJ586 64eaXhiFV3rntgsLavAjW94W596oWmcaL1kjhupsC1ReCh87zMx0opH0_1bx GSsgtIx6TaNYlVRLfl8uJyg7wm_.CS5NeyqUeS0D8KKhwcI1PHnBt5yxvouO ktLa8aIi8y9wPrMHxXRHJ X-Yahoo-SMTP: LAFNfTaswBDguI7meB90l2l3wOU- X-Rocket-Received: from [10.253.183.183] (free7by@117.136.24.71 with xymcookie [106.10.149.123]) by smtp110.mail.bf1.yahoo.com with SMTP; 08 Jan 2014 01:14:41 -0800 PST References: <52CB2627.1000003@bitfrost.no> In-Reply-To: <52CB2627.1000003@bitfrost.no> Mime-Version: 1.0 (1.0) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Message-Id: X-Mailer: iPhone Mail (11B554a) From: by Subject: Re: Strange keyboard mistake Date: Wed, 8 Jan 2014 17:14:29 +0800 To: Hans Petter Selasky Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 09:18:16 -0000 Hi, I am sure that i had not press keys like that, but i think that is not t= he reason. I think it is not because of the bugs in my keyboard neither, since at= that time i got two keyboard connected to my laptop, they behave the same! ----by >> On Jan 7, 2014, at 5:54, Hans Petter Selasky wrote: >>=20 >> On 01/03/14 05:00, by wrote: >> Hi, >> I got a very strange problem. >> I got another keyboard for my laptop, everything goes well for days, b= ut today, for some reasons when in csh environment, i got my new keyboard of= f my laptop's USB port, and just a few minutes later, after i put it back, k= eyboard got a mistake. >> For example, when i type 'b', it became a "smile face", and other key= s became other strange symbols too! >> What is the most strange is that my original keyboard on my laptop be= came the same! >> I got no idea how to do, so i hit Ctrl+Alt+Delete to reboot, after re= boot, everything became normal. >> Does anyone got any ideas about this strange behavior? Or if i do not= want to reboot, what should i do when i encounter this situation again. >> By the way, i use FreeBSD 8.4 RELEASE and my new keyboard is Logitech= K310. >> Thanks. >> ----by >=20 > Hi, >=20 > Are you sure you didn't press any LOCK keys, like NUMLOCK, CAPSLOCK or SCR= OLL LOCK when this happened? >=20 > "usbdump -i usbusX -f Y -vvv -s 65536" will dump the actual traffic toward= s the USB device. Maybe your USB keyboard has some "bugs" in it. >=20 > --HPS >=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"= From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 10:29:48 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43847C8; Wed, 8 Jan 2014 10:29:48 +0000 (UTC) Received: from mail-ob0-x22e.google.com (mail-ob0-x22e.google.com [IPv6:2607:f8b0:4003:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0137D1851; Wed, 8 Jan 2014 10:29:47 +0000 (UTC) Received: by mail-ob0-f174.google.com with SMTP id wn1so1479942obc.5 for ; Wed, 08 Jan 2014 02:29:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=gRNDm2UtBlRbl30IjmSaojexUkJ6h48A7r5gmrVNlP0=; b=mgMZc99Dy/kjXvf3/Qqzr95bW2NrR16lBgaeB1darPYq3JYmGVaspsAxap+KAYJiW/ 1iUmPguie9LL+hkKXCpTkXHZFCRAqDNd0s96umHSOhHtXFEXLJt81FVc68W5n95nyXVr M6CsO7OxKiGkvtXEulb1edbvPWVuxMI01K6gLOf9JDKqIGZx4rvCbSksfs8x3ryZ3q8j LNTmqSup9dZZrez0dPotOWJUmpTjn4YY3WeZMZLSFEwVg1nyyHCc6kPaWo3T7K/403O2 XCJiPqsG5Oq5fF/dknBOswf2MGsFRHZyolC9CGZwpTjOvOuKLq0LXFik+uRngrev+ike V1SQ== MIME-Version: 1.0 X-Received: by 10.60.52.74 with SMTP id r10mr73601oeo.70.1389176987168; Wed, 08 Jan 2014 02:29:47 -0800 (PST) Sender: pali.gabor@gmail.com Received: by 10.182.22.44 with HTTP; Wed, 8 Jan 2014 02:29:47 -0800 (PST) Date: Wed, 8 Jan 2014 11:29:47 +0100 X-Google-Sender-Auth: KEeHuCqI8rhbpHF8LgeEkT9ypPQ Message-ID: Subject: Re: Call for FreeBSD 2013Q4 (October-December) Status Reports From: Gabor Pali To: hackers@freebsd.org, current@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 10:29:48 -0000 Dear FreeBSD Community, Please note that the submission date for the October to December Quarterly Status Reports is January 14th, 2014, a little less than one week away. Please consult my earlier message for the details: On Sat, Dec 14, 2013 at 2:05 PM, Gabor Pali wrote: > They do not have to be very long -- basically they may be about > anything that lets people know what is going on around the FreeBSD > Project. Submission of reports is not restricted to committers: > Anyone who is doing anything interesting and FreeBSD-related can (and > therefore encouraged to) write one! > > The preferred and easiest submission method is to use the XML > generator [1] with the result emailed as an attachment to us, that is, > monthly@FreeBSD.org [2]. There is also an XML template [3] which can > be filled out manually and attached if preferred. For the expected > content and style, please study our guidelines on how to write a good > status reports [4]. > > To enable compilation and publication of the Q4 report as soon as > possible for the January 14th deadline, please be prompt with any > report submissions you may have. > > We are looking forward to all of your 2013Q4 reports! > > Thanks, > Gabor > > > [1] http://www.freebsd.org/cgi/monthly.cgi > [2] mailto:monthly@freebsd.org > [3] http://www.freebsd.org/news/status/report-sample.xml > [4] http://www.freebsd.org/news/status/howto.html From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 12:51:10 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1BF85B74; Wed, 8 Jan 2014 12:51:10 +0000 (UTC) Received: from cargobay.net (cargobay.net [162.220.58.155]) by mx1.freebsd.org (Postfix) with ESMTP id E84531586; Wed, 8 Jan 2014 12:51:09 +0000 (UTC) Received: from [192.168.0.16] (unknown [65.35.151.3]) by cargobay.net (Postfix) with ESMTPSA id 695DD173E; Wed, 8 Jan 2014 12:50:05 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (1.0) Subject: Re: Working on NUMA support From: "Chad J. Milios" X-Mailer: iPhone Mail (11B554a) In-Reply-To: Date: Wed, 8 Jan 2014 07:50:55 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <00E55D89-BDD1-41AD-BBF6-6752B90E8324@ccsys.com> References: To: Andrew Bates X-Mailman-Approved-At: Wed, 08 Jan 2014 13:25:34 +0000 Cc: "freebsd-hackers@freebsd.org" , Adrian Chadd X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 12:51:10 -0000 I'm very interested and excited to watch the progress you are making. Many t= hank yous for grappling this task on FreeBSD! Are you designing/implementing a fully hierarchy-aware approach (nested doma= ins) or is each memory domain separate (not nested, simply "mine" or "not mi= ne" in respect to a particular thread context)? > On Jan 8, 2014, at 3:23 AM, Andrew Bates wrote: >=20 > Hey Adrian, >=20 > We spent the last few months on research/design and plan on spending the > next few months preparing tests, building the physical server(s) to test > on, and finishing the prototype'd functions. >=20 > There is some white-board and pen/paper design that hasn't yet made it to > the github repo, but the prototypes online are hopefully a solid base for > what we intend to complete. In the meantime, we are excited to hear who > else is interested in NUMA and if anyone has suggestions or concerns about= > our approach. >=20 >=20 >> On Wed, Jan 8, 2014 at 12:03 AM, Adrian Chadd wrote:= >>=20 >> Cool! Do you have any working code to implement the API, or is this >> just in the design phase right now? >>=20 >>=20 >> -a >>=20 >>=20 >>> On 6 January 2014 12:11, Andrew Bates wrote: >>> Hey all, >>>=20 >>> My name is Andrew Bates, and I would like to take a bit of your time to >>> talk about NUMA support. >>>=20 >>> Supporting Non-Uniform Memory Access in FreeBSD is something that has >> been >>> brought up in the past < >> http://freebsd.1045724.n5.nabble.com/NUMA-Support-is-there-in-FreeBSD-td4= 865200.html >>> . >>> This is becoming increasingly important now that multiprocessor >>> systems >>> are an expanding technology, thus performance is scaling in terms of cpu= >>> count, rather than just clock rate. >>>=20 >>> There is a great opportunity here to optimize performance. After being >>> asked to look into this by the EMC Isilon Storage Division, myself and a= >>> few colleagues advised by Andrew Pilloud and Jeff Roberson would like to= >>> propose APIs to handle basic memory allocation/management to specific >> NUMA >>> domains. >>>=20 >>> What we have devised so far consists of two levels. First there are the= >>> KPIs, to expose NUMA functionality at a thread level of domain affinity.= >>> Secondly, there would be a userspace/interface to take advantage of the >>> proposed APIs, thus giving users the capability to make their >> applications >>> NUMA-aware. >>>=20 >>> We took the time to look into how many other systems (Linux, Macintosh, >>> Solaris, Windows) already approach this problem, so there are some >> aspects >>> of our solution that are similar to how Linux and Solaris handle NUMA. >>> Unlike Linux libnuma, we are only proposing a few additions and a minima= l >>> library that can easily be expanded later to suit users=E2=80=99 needs. >>>=20 >>>=20 >>> KISS in mind, we came up with the following KPI prototypes >> (freebsdnuma.h) >>> to uncover NUMA in a usable fashion: >>>=20 >>>=20 >>> - >>>=20 >>> cpuset_get_memory_affinity() >>> - >>>=20 >>> cpuset_set_memory_affinity() >>> - >>>=20 >>> move_pages() >>> - >>>=20 >>> migrate_pages() >>> - >>>=20 >>> get_numa_cpus() >>> - >>>=20 >>> get_numa_weights() >>>=20 >>>=20 >>> Then to the second part, we have the following userspace API prototypes >>> (numanor.h) for our interface and testing purposes: >>>=20 >>>=20 >>> - >>>=20 >>> is_numa_available() >>> - >>>=20 >>> set_thread_on_domain() >>> - >>>=20 >>> set_memory_policy() >>> - >>>=20 >>> move_thread() >>>=20 >>>=20 >>> In much much more detail, you can learn more about these prototypes, thi= s >>> project, view our progress, track along, and give input on our github >> repo >>> < https://github.com/andrewbates09/freebsd-numa > or simply via email. >> This >>> repo currently includes fully commented prototypes (like a mini man page= ) >>> and will later include additions to the project. >>>=20 >>> If anyone has any comments, suggestions, concerns, quandaries, or just >>> general thoughts please feel free to contact us, as we would love to hea= r >>> your input! >>>=20 >>> The Leaders: Sakire Arslan Ay, Andrew Pilloud, Jeff Roberson >>> The Team: Andrew Bates, Joshua Clark, Alex Schuldberg, Dustin Walker >>>=20 >>> -- >>> V/Respectfully, >>> Andrew M Bates >>> _______________________________________________ >>> freebsd-hackers@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>> To unsubscribe, send any mail to " >> freebsd-hackers-unsubscribe@freebsd.org" >=20 >=20 >=20 > --=20 > V/Respectfully, > Andrew M Bates > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"= From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 13:29:52 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99E3A93B for ; Wed, 8 Jan 2014 13:29:52 +0000 (UTC) Received: from mail-vb0-x22a.google.com (mail-vb0-x22a.google.com [IPv6:2607:f8b0:400c:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5487618CF for ; Wed, 8 Jan 2014 13:29:52 +0000 (UTC) Received: by mail-vb0-f42.google.com with SMTP id w5so1139291vbf.29 for ; Wed, 08 Jan 2014 05:29:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=K9PD6bz+Cr5yyhMVHwSa60bFUmyUtZDhRLxiitTws+g=; b=LYACJOQmIQTKVhAH3M1Hxl9S7o6x3pEN76NdYz8pA+yRSkxrID2CEo2GcNtgpibwPp 73S3No1QaFStwZC57sgqJPakkwRiC4mVoVCDsf//HThoU2OBnXnMM5iGYyAA/EuYR++w jo4TWxzsXO//Nfc6pQC455/LAOx4YYkGslf2bAzzHNpG1lJkip08aHj1edRWfXks8bxH Uaol4svWE7cFdkmzEBXmQznN6E/nn2JCLb6BSEZqDyiZ8QuKndbuF/1Xxaum6PGsfWLS k4blmXmC8OHtyShP+SjsxUKSPN2bldxnuGTdVoqa82dbOSHlEtLmM36hbLTcYGKMO83k qnug== MIME-Version: 1.0 X-Received: by 10.52.31.227 with SMTP id d3mr2606248vdi.38.1389187790446; Wed, 08 Jan 2014 05:29:50 -0800 (PST) Received: by 10.52.94.51 with HTTP; Wed, 8 Jan 2014 05:29:50 -0800 (PST) In-Reply-To: References: Date: Wed, 8 Jan 2014 15:29:50 +0200 Message-ID: Subject: Re: Continual benchmarking / regression testing? From: Alexander Yerenkow To: Erik Cederstrand Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 13:29:52 -0000 Hello all. I'm working on something which could be used as CI / test tool. I called this FreeBSD as Firmware, or FreeBSD-RO. https://github.com/yerenkow/freebsd-vm-image/tree/master/freebsd-firmware The main idea is that from source tree producing VMDK image, which can be used in VirtualBox / Esxi / * , and to get this image working in readonly via separating configs/data/packages into different disks. I'm planning to extend build system to have snapshotted /usr/obj dirs and make rebuilding even faster. How I see typical workflow: Prepare (or 'cleanup' ) stage: * /zfs/src/src-stable-10 - updated to latest source * /zfs/src/src-stable-10@R300000 - snapshot of revision created * clone created /zfs/src/src-stable-10-clone from snapshot * kernel + world built * /usr/obj snapshot taken; * clone recreated /zfs/src/src-stable-10-clone from latest revision snapshot * /usr/obj cloned from snapshot * optional patches applied to cloned dir * image build (with no cleaning at all, or cleaned only parts touched by patches) * image booted via VBOX * image-OS run tests, upload results/logs * image-OS shut down That use case can be used both to regression testing and to new patches test runs. But, to make that happened, FreeBSD itself should provide some tests to be run. If someone would like to join in developing this - you are very welcome :) -- Regards, Alexander Yerenkow From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 16:38:14 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 69A208B4 for ; Wed, 8 Jan 2014 16:38:14 +0000 (UTC) Received: from mail-we0-x234.google.com (mail-we0-x234.google.com [IPv6:2a00:1450:400c:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 022DD1A4F for ; Wed, 8 Jan 2014 16:38:13 +0000 (UTC) Received: by mail-we0-f180.google.com with SMTP id t61so1685742wes.11 for ; Wed, 08 Jan 2014 08:38:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=/5ia9gUsniA7w3pvtJ5LJz6EXmzjUQudptsqWYdO/OA=; b=W+LxwOlkUkLReUoQ48U3zlCX/QfLYYygpzRhmn/qR4dFoE/CaNvdeDEZZv93+fW37R 1vDgUjY11+AiX2hjGhl+5HCTcV/wcu8vMwjmWlGBkv43qNNEmmKuy7DIyK8789Ho9I1t 5fGBCi+OnUv2Jqfn7pEVZkXM8B75MAiMthOd4OMO5tYzYt58AFWzh+bZXSzBKjvbH7NM bpHPuxYSTIsgxCC8KSpPDc/GCFMaPwh7sSEX1hvSn3o0x05tsHIhVQ2hB95AsJ3ipe9L LdFacrnI2uMopQGMOAy2x5MpXa0jSiBZDV1L32c2yk5ZYUhKH5kcrdXRD9tJGqaM85kG xQCw== MIME-Version: 1.0 X-Received: by 10.180.11.34 with SMTP id n2mr22519755wib.40.1389199092233; Wed, 08 Jan 2014 08:38:12 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Wed, 8 Jan 2014 08:38:12 -0800 (PST) In-Reply-To: References: Date: Wed, 8 Jan 2014 09:38:12 -0700 X-Google-Sender-Auth: nVyig8nUg5xod03PlC5vF8ibeT4 Message-ID: Subject: Re: Continual benchmarking / regression testing? From: Alan Somers To: Erik Cederstrand Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 16:38:14 -0000 On Wed, Jan 8, 2014 at 1:28 AM, Erik Cederstrand wrote: > Den 07/01/2014 kl. 17.09 skrev Ivan Voras : > >> Hello, >> >> Is someone working on a contitual benchmarking / regression testing >> project for FreeBSD? I seem to recall there was a post several months >> ago but I can't find it. > > I have an old project lying with a build/PXE server, a client installer a= nd benchmark runner and a website presenting benchmark reports. It=92s a bi= t outdated, but maybe it can be a starting point? It=92s a mix of shell and= Python. See https://github.com/ecederstrand/perfmon. I like that you stored test results in a SQL database. My SQL-foo is poor, so ATM my framework is using CSV. It also looks like you've got code to generate a website. Do you have any example output? I never got to that part. The PXE stuff, however, does not belong in the benchmark framework, IMHO. I think that the benchmark framework should just include the benchmarking and system profiling aspect, not system configuration. Provisioning and configuring systems can be done in a separate utility, one that can be shared, for example, with the continuous Kyua tester. > > I=92d like to work with you on this, if you want. I'll talk to the boss today to see if I can get any time for benchmarking, or at least to open our proprietary code. -Alan > > Erik > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 20:19:25 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14904362; Wed, 8 Jan 2014 20:19:25 +0000 (UTC) Received: from csmtp11.one.com (csmtp11.one.com [195.47.247.117]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C4D631E1D; Wed, 8 Jan 2014 20:19:24 +0000 (UTC) Received: from bigmac.router9fbd7c.com (unknown [176.222.238.90]) by csmtp11.one.com (Postfix) with ESMTPA id 8FB23C03B555A; Wed, 8 Jan 2014 20:14:00 +0000 (UTC) Received: from bigmac.router9fbd7c.com ([UNAVAILABLE]. [176.222.238.90]) (using TLSv1 with cipher AES128-SHA) by 0.0.0.0:2500 (trex/4.8.87); Wed, 08 Jan 2014 20:14:00 GMT Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Continual benchmarking / regression testing? From: Erik Cederstrand In-Reply-To: Date: Wed, 8 Jan 2014 21:13:59 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <513D4C78-D6FC-45D8-8B1F-CFD2C96E872F@cederstrand.dk> References: To: Alan Somers X-Mailer: Apple Mail (2.1827) Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 20:19:25 -0000 Den 08/01/2014 kl. 17.38 skrev Alan Somers : >=20 > I like that you stored test results in a SQL database. My SQL-foo is > poor, so ATM my framework is using CSV. It also looks like you've got > code to generate a website. Do you have any example output? Yes, there=92s a website to filter results, generate graphs, see commit = messages between two data points and show the hardware and software = configuration of the client running the benchmark. A continuous = benchmarking framework is only useful if it can assist you in analyzing = the data, finding regressions and their cause. > The PXE stuff, however, does not belong in the > benchmark framework, IMHO. I think that the benchmark framework > should just include the benchmarking and system profiling aspect, not > system configuration. Provisioning and configuring systems can be > done in a separate utility, one that can be shared, for example, with > the continuous Kyua tester. System configuration affects benchmark results, so that needs to be = recorded along with the benchmark results. My work was intended as a = complete continuous benchmarking system with a build machine that = produces OS installation images and tells clients what to install and = what to run. But I agree that a benchmark framework contains many = self-contained parts that could be shared among projects. Erik= From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 20:27:53 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6EF3089C; Wed, 8 Jan 2014 20:27:53 +0000 (UTC) Received: from mail-we0-x22f.google.com (mail-we0-x22f.google.com [IPv6:2a00:1450:400c:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D636B1EED; Wed, 8 Jan 2014 20:27:52 +0000 (UTC) Received: by mail-we0-f175.google.com with SMTP id w62so1921304wes.6 for ; Wed, 08 Jan 2014 12:27:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=IwHVh0B/iA6H6kydnVtNARNSAG5zEDQ6lTzo8zm+blc=; b=IsSRzsRUZp57A+patOHt5n+Sy1afRKvJYij4QS5Bo1cjdUeME2H0vZ1LePnFkvn0+0 n/ViuOZvMxfSQq+06fgzQMbQ0fAEOZXt8AOqNm+TN/0n1kuuQQGhsq8qJ/nDtHNBikAJ 0KjcGMro6f48/DmLwIYL1SWCBdkBtYa2XTqEk6raBR3sy0GmpH9k3mVVS6UFuWorAwZM 2k2VfjlJQXUSPUtdc0lhno3qQvVU6i37k4J42C6kt5BihJGf0JuNmesoNI6jJjGGi+5F iO4zDJ7erFChO/VG9ReEt/xoxnkHIxtRrkgouvSdKQdB63zi627qVQm1/Hj/d/D+BrFe kvuQ== MIME-Version: 1.0 X-Received: by 10.194.63.228 with SMTP id j4mr84873252wjs.34.1389212871292; Wed, 08 Jan 2014 12:27:51 -0800 (PST) Sender: asomers@gmail.com Received: by 10.194.22.35 with HTTP; Wed, 8 Jan 2014 12:27:51 -0800 (PST) In-Reply-To: <513D4C78-D6FC-45D8-8B1F-CFD2C96E872F@cederstrand.dk> References: <513D4C78-D6FC-45D8-8B1F-CFD2C96E872F@cederstrand.dk> Date: Wed, 8 Jan 2014 13:27:51 -0700 X-Google-Sender-Auth: U0m-D2tFl20Z1Ljc6PZR8RodxUY Message-ID: Subject: Re: Continual benchmarking / regression testing? From: Alan Somers To: Erik Cederstrand Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-hackers@freebsd.org" , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 20:27:53 -0000 On Wed, Jan 8, 2014 at 1:13 PM, Erik Cederstrand wrote: > Den 08/01/2014 kl. 17.38 skrev Alan Somers : >> >> I like that you stored test results in a SQL database. My SQL-foo is >> poor, so ATM my framework is using CSV. It also looks like you've got >> code to generate a website. Do you have any example output? > > Yes, there=92s a website to filter results, generate graphs, see commit m= essages between two data points and show the hardware and software configur= ation of the client running the benchmark. A continuous benchmarking framew= ork is only useful if it can assist you in analyzing the data, finding regr= essions and their cause. I meant, do you have any generated html that you can share? > >> The PXE stuff, however, does not belong in the >> benchmark framework, IMHO. I think that the benchmark framework >> should just include the benchmarking and system profiling aspect, not >> system configuration. Provisioning and configuring systems can be >> done in a separate utility, one that can be shared, for example, with >> the continuous Kyua tester. > > System configuration affects benchmark results, so that needs to be recor= ded along with the benchmark results. My work was intended as a complete co= ntinuous benchmarking system with a build machine that produces OS installa= tion images and tells clients what to install and what to run. But I agree = that a benchmark framework contains many self-contained parts that could be= shared among projects. My strategy was to separate system configuration from system analysis. That way, the user can configure the system however he likes, using any tools. Then the framework will analyze the system in as much detail as possible. It will determine the CPU type, CPU count, memory type, memory amount, kernel version, network interface configuration, filesystem, filesystem properties, zpool topology, hard disk model, etc. The analysis engine is a lot of work, but its more robust and flexible than tying system configuration into the framework. This separation of analysis from configuration allows the configuration aspect to be handled by a separate tool, that knows nothing of benchmarks. Sequencing the benchmarks can then be run by a short sh script or something. -Alan > > Erik From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 22:59:54 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C990ACAF for ; Wed, 8 Jan 2014 22:59:54 +0000 (UTC) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8EA211A32 for ; Wed, 8 Jan 2014 22:59:54 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 5F1DE3592E2 for ; Wed, 8 Jan 2014 23:59:52 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id 17D6528497; Wed, 8 Jan 2014 23:59:52 +0100 (CET) Date: Wed, 8 Jan 2014 23:59:52 +0100 From: Jilles Tjoelker To: freebsd-hackers@freebsd.org Subject: [patch] libc/resolv: use poll() instead of kqueue() Message-ID: <20140108225951.GB87058@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 22:59:54 -0000 The resolver in libc creates a kqueue for watching a single file descriptor. This can be done using poll() which should be lighter on the kernel and reduce possible problems with rlimits (file descriptors, kqueues). Index: lib/libc/include/port_before.h =================================================================== --- lib/libc/include/port_before.h (revision 260048) +++ lib/libc/include/port_before.h (working copy) @@ -5,7 +5,7 @@ #define _LIBC 1 #define DO_PTHREADS 1 -#define USE_KQUEUE 1 +#define USE_POLL 1 #define ISC_SOCKLEN_T socklen_t #define ISC_FORMAT_PRINTF(fmt, args) \ Index: lib/libc/resolv/res_send.c =================================================================== --- lib/libc/resolv/res_send.c (revision 260048) +++ lib/libc/resolv/res_send.c (working copy) @@ -77,7 +77,7 @@ __FBSDID("$FreeBSD$"); */ #include "port_before.h" -#ifndef USE_KQUEUE +#if !defined(USE_KQUEUE) && !defined(USE_POLL) #include "fd_setsize.h" #endif @@ -963,7 +963,7 @@ send_dg(res_state statp, timeout.tv_nsec/1000000; pollfd.fd = s; pollfd.events = POLLRDNORM; - n = poll(&pollfd, 1, polltimeout); + n = _poll(&pollfd, 1, polltimeout); #endif /* USE_POLL */ if (n == 0) { From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 9 21:08:41 2014 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 15771AEC for ; Thu, 9 Jan 2014 21:08:41 +0000 (UTC) Received: from mail-la0-x234.google.com (mail-la0-x234.google.com [IPv6:2a00:1450:4010:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8E8571ED4 for ; Thu, 9 Jan 2014 21:08:40 +0000 (UTC) Received: by mail-la0-f52.google.com with SMTP id y1so2621189lam.11 for ; Thu, 09 Jan 2014 13:08:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=B0UJXGiAYS7/L7HJAcuLI+J3ebW/fRZgTib7Eje4V4Q=; b=X9Ae7gEH/8BWRlQN/p665NNSs1IOQUa0eXH8GwaVurGJC7IKqJVCpo6wtfndgtYbJb Gou9hvc8q6cTWyGG1msuFuwhwX2kM8qdXZoXI7uWw3yg8bkdz1O5JPACeL5K557yzSmk 55MLazK1xUDLMT9fjJEqbFMnWA2wd9R6q6b4GVH9zpkqytwZGTiCROSygf1faestE6/1 /3rWCugEC7Owh7j4jHptoAvQ4udIlQHs1g9g4FmC3g9DG/7owKfnieXpou9tgC8PQgls lDYWkw2p0utz/cG2EfExyl7IL0nHeRA5nTVWOOwNKM6YhDtrf1Igo/VT5o2hsKt5xDmU vIkQ== X-Received: by 10.152.115.130 with SMTP id jo2mr2303866lab.2.1389301718376; Thu, 09 Jan 2014 13:08:38 -0800 (PST) Received: from [10.0.1.20] ([176.193.50.79]) by mx.google.com with ESMTPSA id e10sm2515289laa.6.2014.01.09.13.08.36 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 09 Jan 2014 13:08:37 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: FreeBSD-10 and processes swapping out for no reason From: Dmitry Sivachenko In-Reply-To: Date: Fri, 10 Jan 2014 01:08:35 +0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: hackers@freebsd.org X-Mailer: Apple Mail (2.1827) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jan 2014 21:08:41 -0000 On 07 =D1=8F=D0=BD=D0=B2. 2014 =D0=B3., at 20:16, Dmitry Sivachenko = wrote: > Hello! >=20 > Recently I updated several of my servers from 9.2-STABLE 10.0: > 10.0-PRERELEASE FreeBSD 10.0-PRERELEASE #0 r259661 >=20 > Workload did not change at all, same programs, same load, same = hardware. >=20 > I noticed that some of the processes on 10.0 boxes are marked as = swapped out in top(1) output: >=20 > 1436 root 1 43 0 16524K 0K nanslp 14 1:14 0.00% = > 1381 smmsp 1 20 0 23988K 0K pause 18 0:04 0.00% = 99348 mitya 1 21 0 23492K 0K pause 16 0:00 = 0.00% >=20 > ps also shows them as swapped out (W as second character in state = field): > 1381 - IWs 0:00.00 sendmail: Queue runner@00:30:00 for = /var/spool/clie > 1436 - IWs 0:00.00 /usr/sbin/cron -s > 80231 - IWs 0:00.00 /usr/local/sbin/collectdmon -c = /usr/local/sbin/coll > 99348 1 IWs 0:00.00 -csh (csh) >=20 > These machines have enought RAM and that never happened on 9.2-STABLE. >=20 > I turned off swap: > # swapinfo > Device 1K-blocks Used Avail Capacity > # >=20 > And state of these processes did not change: still second characted in = state field of ps output is W, and top shows them in angles. >=20 > How should I treat that? >=20 > Thanks. Anyone else observing this? Right now looks like I can't trust top(1) = values for RES and ps(1) states. From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 10 05:29:16 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 97F07B06 for ; Fri, 10 Jan 2014 05:29:16 +0000 (UTC) Received: from elektropost.org (elektropost.org [217.115.13.199]) by mx1.freebsd.org (Postfix) with ESMTP id D57C416D6 for ; Fri, 10 Jan 2014 05:29:15 +0000 (UTC) Received: (qmail 53744 invoked from network); 10 Jan 2014 05:29:07 -0000 Received: from elektropost.org (HELO elektropost.org) (erdgeist@erdgeist.org) by elektropost.org with CAMELLIA256-SHA encrypted SMTP; 10 Jan 2014 05:29:07 -0000 Message-ID: <52CF850A.9060906@erdgeist.org> Date: Fri, 10 Jan 2014 06:28:42 +0100 From: Dirk Engling User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: FreeBSD Hackers Subject: Atom Board ACPI API MOPNV10J failing since 9.1 X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Jan 2014 05:29:16 -0000 Dear fellow hackers, after upgrade my atom board to use the newer zfs features, I could not use my network cards anymore, neither the builtin realtek, nor my intel e1000 card. You can see it in the http://erdgeist.org/MOPNV10J/dmesg (here one from FreeBSD-10.0-RC5, but 9.1 and 9.2 behave the same) re0: PHY read failed re0: attaching PHYs failed ... em0: port 0x1000-0x103f irq 21 at device 0.0 on pci5 em0: Setup of Shared code failed device_attach: em0 attach returned 6 Searching the forums and mailing lists I found this might be related to the new pcib code and indeed I found in every dmesg since 9.1: pcib1: at device 28.0 on pci0 pcib1: failed to allocate initial prefetch window: 0xe0000000-0xe00fffff >From now I'm out of my depths, if someone can help me with some pointers on whom to ask, it would be appreciated. For completeness, the output of devinfo -r http://erdgeist.org/MOPNV10J/devinfo and of acpidump -dt http://erdgeist.org/MOPNV10J/acpidump. Regards and TIA, erdgeist From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 10 10:10:32 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BF86E32F for ; Fri, 10 Jan 2014 10:10:32 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7A3C21CC8 for ; Fri, 10 Jan 2014 10:10:32 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1W1Z2p-0000xN-6w for freebsd-hackers@freebsd.org; Fri, 10 Jan 2014 11:10:27 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 10 Jan 2014 11:10:27 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 10 Jan 2014 11:10:27 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Subject: Wiki - What's new for FreeBSD 11 Date: Fri, 10 Jan 2014 11:10:16 +0100 Lines: 31 Message-ID: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="h2Ahw6sUwIdfAiOtkk7AMHg3RttTcdbFN" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 X-Enigmail-Version: 1.6 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Jan 2014 10:10:32 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --h2Ahw6sUwIdfAiOtkk7AMHg3RttTcdbFN Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hello, In the anticipation of FreeBSD 10 RELEASE, I've started the wiki page for FreeBSD 11: https://wiki.freebsd.org/WhatsNew/FreeBSD11 You know what to do :) The page is structured identically to the one for FreeBSD 10 (https://wiki.freebsd.org/WhatsNew/FreeBSD10). --h2Ahw6sUwIdfAiOtkk7AMHg3RttTcdbFN Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iKYEARECAGYFAlLPxwhfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDYxNDE4MkQ3ODMwNDAwMDJFRUIzNDhFNUZE MDhENTA2M0RGRjFEMkMACgkQ/QjVBj3/HSxQoQCdEoZ2i9ac1HjGNl6/HZa+gXgQ xbIAn1BJcfBK+e/9WdNMudF+92rOfHEt =fzXT -----END PGP SIGNATURE----- --h2Ahw6sUwIdfAiOtkk7AMHg3RttTcdbFN-- From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 10 21:42:44 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CCCF036C; Fri, 10 Jan 2014 21:42:44 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id A83B212E7; Fri, 10 Jan 2014 21:42:44 +0000 (UTC) Received: from Alfreds-MacBook-Pro.local (unknown [216.38.154.18]) by elvis.mu.org (Postfix) with ESMTPSA id D9F4B1A3C2D; Fri, 10 Jan 2014 13:42:43 -0800 (PST) Message-ID: <52D06954.9000304@freebsd.org> Date: Fri, 10 Jan 2014 13:42:44 -0800 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Alan Cox , Hans Petter Selasky , Neel Natu , FreeBSD Hackers Subject: usb + other drivers stop working on 128GB+ memory machines References: <50BDB148.1060607@mu.org> In-Reply-To: <50BDB148.1060607@mu.org> X-Forwarded-Message-Id: <50BDB148.1060607@mu.org> Content-Type: multipart/mixed; boundary="------------010602060907090305070104" X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Tommy Stiansen X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Jan 2014 21:42:45 -0000 This is a multi-part message in MIME format. --------------010602060907090305070104 Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit Hey Alan, Neel and Hans, We're testing FreeBSD 10 here and still having problems, once we go over 128GB of memory then USB stops working. When we artificially limit memory to 128GB or lower we are OK. Is there any chance we can revisit this patch so that large memory systems don't use up the lower memory space which seems to be needed by some drivers? I'm having a bit of trouble explaining to people that too much memory == no keyboard on FreeBSD. I have the patch that seemed to work for us before. Any chance this can go into FreeBSD soon? -Alfred -------- Original Message -------- Subject: Re: Questions about FreeBSD amd64 memory layout. Date: Tue, 04 Dec 2012 00:16:08 -0800 From: Alfred Perlstein To: Alan Cox CC: Alan Cox , Xin LI On 12/3/12 11:23 PM, Alan Cox wrote: > On 12/03/2012 18:15, Alfred Perlstein wrote: >> Hello Alan, >> >> The other day I ran a copy of FreeBSD 9.1 with my maxusers patches >> (from current). >> >> The machine had 256 gigs of RAM. >> >> Due to that much memory, maxusers was upwards of 24860. >> >> What then happened was that the mfi driver, and I think also the USB >> driver would not work. >> >> The mfi driver stopped working because it got the following error: >> mfi0: Cannot allocate verbuf_h_dmamap memory >> >> This appears to be due to this in the mfi driver: >>> /* Start: LSIP200113393 */ >>> if (bus_dma_tag_create( sc->mfi_parent_dmat, /* parent */ >>> 1, 0, /* algnmnt, >>> boundary */ >>> BUS_SPACE_MAXADDR_32BIT,/* lowaddr */ >>> BUS_SPACE_MAXADDR, /* highaddr */ >>> NULL, NULL, /* filter, >>> filterarg */ >>> MEGASAS_MAX_NAME*sizeof(bus_addr_t), /* maxsize */ >>> 1, /* msegments */ >>> MEGASAS_MAX_NAME*sizeof(bus_addr_t), /* maxsegsize */ >>> 0, /* flags */ >>> NULL, NULL, /* lockfunc, >>> lockarg */ >>> &sc->verbuf_h_dmat)) { >>> device_printf(sc->mfi_dev, "Cannot allocate >>> verbuf_h_dmat DMA tag\n"); >>> return (ENOMEM); >>> } >>> if (bus_dmamem_alloc(sc->verbuf_h_dmat, (void **)&sc->verbuf, >>> BUS_DMA_NOWAIT, &sc->verbuf_h_dmamap)) { >>> device_printf(sc->mfi_dev, "Cannot allocate >>> verbuf_h_dmamap memory\n"); >> What I'm thinking is happening is that by the time we get to mfi >> driver enough of the below 4GB memory is used up by callout wheels, >> nbufs, various hash tables, etc that we wind up unable to get memory >> in this region. >> >> This could (and probably is) a wrong assumption, but it's what makes >> sense to me right now. >> > > I can believe it, or more precisely I know of nothing that immediately > disproves it. > > >> I'm wondering how the kernel map gets populated, and if it would be >> possible, and if it would be advisable to change the allocation >> strategy to come from the tail end of physical memory instead of the >> front. >> > > There is no intentional "allocation strategy" in the sense that you are > using the phrase here. Much of the VM system, including the physical > memory allocator, is initialized early in the boot process, in fact, > before callout wheels, nbufs, etc. are allocated. So, the standard > physical memory allocator is being used for callout wheels, nbufs, etc., > and this allocator takes pages from the cache/free page queues in > whatever arbitrary order they happen to be in. I can believe that we > currently initialize the cache/free page queues in an order that results > in the allocation of pages from low physical addresses first. > > The physical memory allocator does, however, have a way of dealing with > low physical address ranges that you don't want to allocate from except > explicitly, e.g., contigmalloc()/kmem_alloc_contig(), or as a last > resort. This is currently only used for the physical address range for > ISA DMA. > > I've attached a patch that abuses the ISA DMA range, extending it to > 4GB. See if this patch enables you to boot. > > It does! Everything is fixed now. What now? Can I help somehow? ~ % sysctl -a| grep maxuser kern.maxusers: 33049 ~ % dmesg| grep mfi mfi0: port 0x8000-0x80ff mem 0xc7a60000-0xc7a63fff,0xc7a00000-0xc7a3ffff irq 26 at device 0.0 on pci1 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23 mfi0: MaxCmd = 3f0 MaxSgl = 46 state = b75003f0 mfi0: 1436 (407894536s/0x0020/info) - Shutdown command received from host mfi0: 1437 (boot + 4s/0x0020/info) - Firmware initialization started (PCI ID 005b/1000/0690/15d9) mfi0: 1438 (boot + 4s/0x0020/info) - Firmware version 3.190.05-1669 mfi0: 1439 (boot + 5s/0x0020/info) - Package version 23.7.0-0029 mfi0: 1440 (boot + 5s/0x0020/info) - Board Revision mfi0: 1441 (boot + 25s/0x0002/info) - Inserted: PD 10(e0xfc/s0) mfi0: 1442 (boot + 25s/0x0002/info) - Inserted: PD 10(e0xfc/s0) Info: enclPd=fc, scsiType=0, portMap=00, sasAddr=4433221103000000,0000000000000000 mfi0: 1443 (boot + 26s/0x0001/info) - Policy change on VD 00/0 to [ID=00,dcp=65,ccp=64,ap=0,dc=0] from [ID=00,dcp=65,ccp=65,ap=0,dc=0] mfi0: 1444 (407894583s/0x0020/info) - Time established as 12/04/12 0:03:03; (37 seconds since power on) mfi0: 1445 (407894819s/0x0020/info) - Host driver is loaded and operational mfid0 on mfi0 mfid0: 2861022MB (5859373056 sectors) RAID volume (no label) is optimal Trying to mount root from ufs:/dev/mfid0p2 [rw]... --------------010602060907090305070104 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="hack2.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="hack2.patch" Index: amd64/include/vmparam.h =================================================================== --- amd64/include/vmparam.h (revision 243366) +++ amd64/include/vmparam.h (working copy) @@ -106,10 +106,13 @@ * accessible by ISA DMA and VM_FREELIST_ISADMA is for physical pages * that are below that address. */ -#define VM_NFREELIST 2 -#define VM_FREELIST_DEFAULT 0 -#define VM_FREELIST_ISADMA 1 +#define VM_NFREELIST 3 +#define VM_FREELIST_HIGHMEM 0 +#define VM_FREELIST_DEFAULT 1 +#define VM_FREELIST_ISADMA 2 +#define VM_HIGHMEM_ADDRESS ((vm_paddr_t)1 << 32) + /* * An allocation size of 16MB is supported in order to optimize the * use of the direct map by UMA. Specifically, a cache line contains --------------010602060907090305070104-- From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 10 22:11:54 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C7171A8D; Fri, 10 Jan 2014 22:11:54 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 537831560; Fri, 10 Jan 2014 22:11:54 +0000 (UTC) Received: from [10.234.62.188] (184.sub-174-240-38.myvzw.com [174.240.38.184]) by elvis.mu.org (Postfix) with ESMTPSA id E6FF11A3C55; Fri, 10 Jan 2014 14:11:52 -0800 (PST) References: <50BDB148.1060607@mu.org> Mime-Version: 1.0 (1.0) In-Reply-To: Message-Id: X-Mailer: iPhone Mail (11B554a) From: Alfred Perlstein Subject: Re: usb + other drivers stop working on 128GB+ memory machines Date: Fri, 10 Jan 2014 14:11:47 -0800 To: Hans Petter Selasky Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Tommy Stiansen , FreeBSD Hackers , Alfred Perlstein , Neel Natu , Alan Cox X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Jan 2014 22:11:54 -0000 > On Jan 10, 2014, at 2:09 PM, Hans Petter Selasky wrote: >=20 > Hi, >=20 > The newer XHCI chipset does support full 64-bit ranges, if the HW guys did= their job. We've seen in the past 32-bit hardware being cut down to 2GB of R= AM in the hardware because some OS'es don't support more :-) >=20 > I think in general that keeping DMA buffers below 4GB is a good idea. Wasn= 't the allocator changed some years back to allocate from the top of memory i= nstead of the bottom? It appears not. Or maybe it got reverted? When you have so much ram simply stealing some % unconditionally might be ok= so long as devices run.=20 Note some devices have 24 but limits even! >=20 > --HPS > =20 > -----Original message----- > > From:Alfred Perlstein > > Sent: Friday 10th January 2014 22:43 > > To: Alan Cox ; Hans Petter Selasky ; Neel Natu ; FreeBSD Hackers > > Cc: Tommy Stiansen > > Subject: usb + other drivers stop working on 128GB+ memory machines > >=20 > > Hey Alan, Neel and Hans, > >=20 > > We're testing FreeBSD 10 here and still having problems, once we go over= =20 > > 128GB of memory then USB stops working. When we artificially limit=20 > > memory to 128GB or lower we are OK. > >=20 > > Is there any chance we can revisit this patch so that large memory=20 > > systems don't use up the lower memory space which seems to be needed by=20= > > some drivers? > >=20 > > I'm having a bit of trouble explaining to people that too much memory =3D= =3D=20 > > no keyboard on FreeBSD. > >=20 > > I have the patch that seemed to work for us before. Any chance this can= =20 > > go into FreeBSD soon? > >=20 > >=20 > >=20 > > -Alfred > >=20 > >=20 > > -------- Original Message -------- > > Subject: Re: Questions about FreeBSD amd64 memory layout. > > Date: Tue, 04 Dec 2012 00:16:08 -0800 > > From: Alfred Perlstein > > To: Alan Cox > > CC: Alan Cox , Xin LI > >=20 > >=20 > >=20 > > On 12/3/12 11:23 PM, Alan Cox wrote: > > > On 12/03/2012 18:15, Alfred Perlstein wrote: > > >> Hello Alan, > > >> > > >> The other day I ran a copy of FreeBSD 9.1 with my maxusers patches > > >> (from current). > > >> > > >> The machine had 256 gigs of RAM. > > >> > > >> Due to that much memory, maxusers was upwards of 24860. > > >> > > >> What then happened was that the mfi driver, and I think also the USB > > >> driver would not work. > > >> > > >> The mfi driver stopped working because it got the following error: > > >> mfi0: Cannot allocate verbuf_h_dmamap memory > > >> > > >> This appears to be due to this in the mfi driver: > > >>> /* Start: LSIP200113393 */ > > >>> if (bus_dma_tag_create( sc->mfi_parent_dmat, /* parent *= / > > >>> 1, 0, /* algnmnt,= > > >>> boundary */ > > >>> BUS_SPACE_MAXADDR_32BIT,/* lowaddr *= / > > >>> BUS_SPACE_MAXADDR, /* highaddr= */ > > >>> NULL, NULL, /* filter, > > >>> filterarg */ > > >>> MEGASAS_MAX_NAME*sizeof(bus_addr_t), /* maxsize *= / > > >>> 1, /* msegment= s */ > > >>> MEGASAS_MAX_NAME*sizeof(bus_addr_t), /* maxsegsiz= e */ > > >>> 0, /* flags */= > > >>> NULL, NULL, /* lockfunc= , > > >>> lockarg */ > > >>> &sc->verbuf_h_dmat)) { > > >>> device_printf(sc->mfi_dev, "Cannot allocate > > >>> verbuf_h_dmat DMA tag\n"); > > >>> return (ENOMEM); > > >>> } > > >>> if (bus_dmamem_alloc(sc->verbuf_h_dmat, (void **)&sc->verbu= f, > > >>> BUS_DMA_NOWAIT, &sc->verbuf_h_dmamap)) { > > >>> device_printf(sc->mfi_dev, "Cannot allocate > > >>> verbuf_h_dmamap memory\n"); > > >> What I'm thinking is happening is that by the time we get to mfi > > >> driver enough of the below 4GB memory is used up by callout wheels, > > >> nbufs, various hash tables, etc that we wind up unable to get memory > > >> in this region. > > >> > > >> This could (and probably is) a wrong assumption, but it's what makes > > >> sense to me right now. > > >> > > > > > > I can believe it, or more precisely I know of nothing that immediately= > > > disproves it. > > > > > > > > >> I'm wondering how the kernel map gets populated, and if it would be > > >> possible, and if it would be advisable to change the allocation > > >> strategy to come from the tail end of physical memory instead of the > > >> front. > > >> > > > > > > There is no intentional "allocation strategy" in the sense that you ar= e > > > using the phrase here. Much of the VM system, including the physical > > > memory allocator, is initialized early in the boot process, in fact, > > > before callout wheels, nbufs, etc. are allocated. So, the standard > > > physical memory allocator is being used for callout wheels, nbufs, etc= ., > > > and this allocator takes pages from the cache/free page queues in > > > whatever arbitrary order they happen to be in. I can believe that we > > > currently initialize the cache/free page queues in an order that resul= ts > > > in the allocation of pages from low physical addresses first. > > > > > > The physical memory allocator does, however, have a way of dealing wit= h > > > low physical address ranges that you don't want to allocate from excep= t > > > explicitly, e.g., contigmalloc()/kmem_alloc_contig(), or as a last > > > resort. This is currently only used for the physical address range fo= r > > > ISA DMA. > > > > > > I've attached a patch that abuses the ISA DMA range, extending it to > > > 4GB. See if this patch enables you to boot. > > > > > > > > It does! Everything is fixed now. > >=20 > > What now? Can I help somehow? > >=20 > > =CB=9C % sysctl -a| grep maxuser > > kern.maxusers: 33049 > > =CB=9C % dmesg| grep mfi > > mfi0: port 0x8000-0x80ff mem > > 0xc7a60000-0xc7a63fff,0xc7a00000-0xc7a3ffff irq 26 at device 0.0 on pci1= > > mfi0: Using MSI > > mfi0: Megaraid SAS driver Ver 4.23 > > mfi0: MaxCmd =3D 3f0 MaxSgl =3D 46 state =3D b75003f0 > > mfi0: 1436 (407894536s/0x0020/info) - Shutdown command received from hos= t > > mfi0: 1437 (boot + 4s/0x0020/info) - Firmware initialization started > > (PCI ID 005b/1000/0690/15d9) > > mfi0: 1438 (boot + 4s/0x0020/info) - Firmware version 3.190.05-1669 > > mfi0: 1439 (boot + 5s/0x0020/info) - Package version 23.7.0-0029 > > mfi0: 1440 (boot + 5s/0x0020/info) - Board Revision > > mfi0: 1441 (boot + 25s/0x0002/info) - Inserted: PD 10(e0xfc/s0) > > mfi0: 1442 (boot + 25s/0x0002/info) - Inserted: PD 10(e0xfc/s0) Info: > > enclPd=3Dfc, scsiType=3D0, portMap=3D00, sasAddr=3D4433221103000000,0000= 000000000000 > > mfi0: 1443 (boot + 26s/0x0001/info) - Policy change on VD 00/0 to > > [ID=3D00,dcp=3D65,ccp=3D64,ap=3D0,dc=3D0] from [ID=3D00,dcp=3D65,ccp=3D6= 5,ap=3D0,dc=3D0] > > mfi0: 1444 (407894583s/0x0020/info) - Time established as 12/04/12 > > 0:03:03; (37 seconds since power on) > > mfi0: 1445 (407894819s/0x0020/info) - Host driver is loaded and operatio= nal > > mfid0 on mfi0 > > mfid0: 2861022MB (5859373056 sectors) RAID volume (no label) is optimal > > Trying to mount root from ufs:/dev/mfid0p2 [rw]... > >=20 > >=20 > >=20 > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.or= g"