From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 30 05:50:10 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0BDEC106567A for ; Sun, 30 Sep 2012 05:50:10 +0000 (UTC) (envelope-from jguojun@sbcglobal.net) Received: from nm14-vm0.bullet.mail.sp2.yahoo.com (nm14-vm0.bullet.mail.sp2.yahoo.com [98.139.91.246]) by mx1.freebsd.org (Postfix) with SMTP id 1A6198FC1F for ; Sun, 30 Sep 2012 05:50:03 +0000 (UTC) Received: from [72.30.22.93] by nm14.bullet.mail.sp2.yahoo.com with NNFMP; 30 Sep 2012 05:49:57 -0000 Received: from [98.139.44.89] by tm15.bullet.mail.sp2.yahoo.com with NNFMP; 30 Sep 2012 05:49:57 -0000 Received: from [127.0.0.1] by omp1026.access.mail.sp2.yahoo.com with NNFMP; 30 Sep 2012 05:49:57 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 366535.74119.bm@omp1026.access.mail.sp2.yahoo.com Received: (qmail 39815 invoked by uid 60001); 30 Sep 2012 05:49:56 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sbcglobal.net; s=s1024; t=1348984196; bh=YiwQWfvxWsp1stoutLoImftzYzhQPymriWyuS+mM1dc=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:Cc:MIME-Version:Content-Type; b=P/jlutymmw3cg3lQF/aSwIVWa55lL6p6ssGegfmR9tba6oMiTEOiXwZtOZ+bY8ROxSisnc2z9gr/AkiRKCh917sq7kBegrjSoO55nBaVaMcFjqVOBC8H+qIfi/TyaOqfKWkO1VjNd46l2LfMElL/oyZQdXebJ+zfVtzHFXZQ6tk= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=sbcglobal.net; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:Cc:MIME-Version:Content-Type; b=46PE/iuFpfbI/GzmbIuARs8d/Isuah+qsjNF8MHcfh4M/FnFZ0JdU4lsHWl/b7kaQEbXPbjbl+VeHyNgv9D6FdoeJiCjfV5On14Qbg9cUaHgGdoQrn4W0IczYI0VsJMhgtpxCJWRWI4k8Vie3R/MtjkjWf+k6MyXPmsD9sFTI8k=; X-YMail-OSG: kSrPtgsVM1nKsPKHmxxDXJpYKe.YQs5Kbx4MHSqZmNuajOF 1ZYtvsvQIeHvw6h6spX4Yts9goE8NNj48gE3Ucbb16w2g23jPU_hBdK0DVO2 UbwHkGZd4xQXf03yzI2scGnrJqahLUJaLzjfCap2DD3H2ShxONc2fPebUjbt xMcH6rZDiu33c1hj8_JzPKM3etebjTB9MrIdcp6RTW8m1LHbvctzeNK32rs3 hIkorC_RKuy4AcxAqn0vAFjh3b.3AVbr_5UbJcsPzkds82C9T2L2vPS7rRD1 _wz7nOx5uv0dzBIV6j_5r9cdqlBctR.nxgQj5jHr4hxNw_ZJ2LADp0mwtRJm WvtofBbPQu4VVgZYEw9ic.4ltA4A6vFLtysfTsoccups56ydvhWixFIQCBjr _JCOo_IhBNzoVom8HcULBmxcJ.BqNl02ijf__EoBv8t3vdkRY5dJFV5KWbPf fyTx8wOyeaYrvjep2lCKuxyIJaIDoMr3zautKXnNxC36qLMXZ.2RwhxZMAFE VfNzrUJ0- Received: from [75.37.1.111] by web180906.mail.ne1.yahoo.com via HTTP; Sat, 29 Sep 2012 22:49:56 PDT X-Mailer: YahooMailRC/708 YahooMailWebService/0.8.121.434 Message-ID: <1348984196.37002.YahooMailRC@web180906.mail.ne1.yahoo.com> Date: Sat, 29 Sep 2012 22:49:56 -0700 (PDT) From: Jin Guojun To: questions@freebsd.org MIME-Version: 1.0 X-Mailman-Approved-At: Sun, 30 Sep 2012 11:51:18 +0000 Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: hackers@freebsd.org Subject: system hangs during dump + compress > usb2-drive X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Sep 2012 05:50:10 -0000 In FreeBSD 8.3 release (possibly in earlier release), dump a file system has 2-3GB or more content can cause system hang in a specific case (pipe to compression): dump FS-on-SATA-drive > usb-drive OK dump FS-on-SATA-drive | anyCompress > sata-drive OK mv a-large-dump-file from STAT drive to a USB drive OK dump small-FS-on-SATA-drive | anyCompress > usb-drive OK small -- 1.8GB or less dump large-FS-on-SATA-drive | anyCompress > usb-drive hang content is 3GB or larger (did not try around 2GB yet) When system hangs, no sub system, such video, network, etc, will function. Typically, the unfinished compressed dump file is around 1.5-2.7GB, so guessing dumped file content is close to or over 2GB when failure occurred. Has anyone encountered the same problem? Because this usually takes a few hours to occur, this is hard to watch how/when it happens. Is any way to debug or determine what status the system is? -Jin From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 30 19:32:11 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 29B191065674 for ; Sun, 30 Sep 2012 19:32:11 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe01.c2i.net [212.247.154.2]) by mx1.freebsd.org (Postfix) with ESMTP id A8AB98FC18 for ; Sun, 30 Sep 2012 19:32:10 +0000 (UTC) X-T2-Spam-Status: No, hits=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 Received: from [176.74.213.204] (account mc467741@c2i.net HELO laptop015.hselasky.homeunix.org) by mailfe01.swip.net (CommuniGate Pro SMTP 5.4.4) with ESMTPA id 328477665; Sun, 30 Sep 2012 21:32:03 +0200 From: Hans Petter Selasky To: freebsd-hackers@freebsd.org, geoffrey levand Date: Sun, 30 Sep 2012 21:33:31 +0200 User-Agent: KMail/1.13.7 (FreeBSD/9.1-PRERELEASE; KDE/4.8.4; amd64; ; ) References: <1348410653.373800982@f180.mail.ru> <201209241820.07558.hselasky@c2i.net> <1348910721.385146900@f89.mail.ru> In-Reply-To: <1348910721.385146900@f89.mail.ru> X-Face: 'mmZ:T{)),Oru^0c+/}w'`gU1$ubmG?lp!=R4Wy\ELYo2)@'UZ24N@d2+AyewRX}mAm; Yp |U[@, _z/([?1bCfM{_"B<.J>mICJCHAzzGHI{y7{%JVz%R~yJHIji`y>Y}k1C4TfysrsUI -%GU9V5]iUZF&nRn9mJ'?&>O MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201209302133.31641.hselasky@c2i.net> Cc: Subject: Re: How to claim only some of USB interfaces of a composite USB device X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Sep 2012 19:32:11 -0000 On Saturday 29 September 2012 11:25:21 geoffrey levand wrote: > geoffrey levand Can you verify this patch: http://svn.freebsd.org/changeset/base/241078 --HPS From owner-freebsd-hackers@FreeBSD.ORG Sun Sep 30 20:07:38 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16D21106566C; Sun, 30 Sep 2012 20:07:38 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id EAF928FC0A; Sun, 30 Sep 2012 20:07:37 +0000 (UTC) Received: from Xins-MacBook-Pro.local (c-67-188-85-47.hsd1.ca.comcast.net [67.188.85.47]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 9AE651CEEB; Sun, 30 Sep 2012 13:07:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1349035657; bh=R4gkWUbdrcUay0TwKDt2TkgXjp7gLugLGSzDmdxkCzk=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=gmYwDZ1vaSYscBSfYzZbtopZI+5uK4rAQgyTPOj9yl43FHrkYMM09MGAzMKJ9+0qV f9lGr3WMTbRnc+FlDwRZwbVRAphhJHYDug+GWsSkivlYfxZiyvnhiJwYjkDKHEPa+S M3qJT0DgYCglUAyLLlxbV3a9MfdaNT1J+qrkB6oE= Message-ID: <5068A688.4070100@delphij.net> Date: Sun, 30 Sep 2012 13:07:36 -0700 From: Xin Li Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 To: Jin Guojun References: <1348984196.37002.YahooMailRC@web180906.mail.ne1.yahoo.com> In-Reply-To: <1348984196.37002.YahooMailRC@web180906.mail.ne1.yahoo.com> X-Enigmail-Version: 1.4.4 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: hackers@freebsd.org, questions@freebsd.org Subject: Re: system hangs during dump + compress > usb2-drive X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Sep 2012 20:07:38 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 9/29/12 10:49 PM, Jin Guojun wrote: > In FreeBSD 8.3 release (possibly in earlier release), dump a file > system has 2-3GB or more content can cause system hang in a > specific case (pipe to compression): > > dump FS-on-SATA-drive > usb-drive OK dump FS-on-SATA-drive > | anyCompress > sata-drive OK mv a-large-dump-file from > STAT drive to a USB drive OK dump small-FS-on-SATA-drive | > anyCompress > usb-drive OK small -- 1.8GB or less dump > large-FS-on-SATA-drive | anyCompress > usb-drive hang > content is 3GB or larger (did not try around 2GB yet) > > When system hangs, no sub system, such video, network, etc, will > function. Typically, the unfinished compressed dump file is around > 1.5-2.7GB, so guessing dumped file content is close to or over 2GB > when failure occurred. > > Has anyone encountered the same problem? > > Because this usually takes a few hours to occur, this is hard to > watch how/when it happens. Is any way to debug or determine what > status the system is? For starters I'd use a different console for doing procstat -kk -a and see what the system is doing. (Perhaps also top) I *think* that if it's just hanging for some time, it's probably because the system is trying to take a snapshot? It takes time on UFS when creating and removing the snapshot. Just a guess... Cheers, -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) iQEcBAEBCAAGBQJQaKaIAAoJEG80Jeu8UPuzfH4IAL3k2M/KHV39FrI0U4lZ1yu/ bFbJJubQHzjfNbDrI4er1Xg6S0sN0DNnRoD/bQFKKHvQpfqcCUOwUtpq0kssyfLY 4XQOF9nhcyvL/INz6ArtI7EhKh/2cADb+1zp+NMsFyqvn3F09VPvx6h9z6ufaian LlAA6uisZSl/eGv5uNGGcudiUxSALql8UniZVHJvyO+pCjOAwL+MBxfqQ4LW3DEy ngvkvCeQ2nK/k0oQDq5jt9A9+D+7b3+Wo+4sMkIN7uTMgPpET4JSgWgxkzG1xM+l VMTAUHiqdeKp2JbWot1sRE5K7SiLPDmVGXEa0w+duBbvtuk8M/uXLootky0y1Oc= =zTBM -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 1 00:09:18 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CF4B1106564A; Mon, 1 Oct 2012 00:09:18 +0000 (UTC) (envelope-from prvs=16217704b1=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 3B4CF8FC0C; Mon, 1 Oct 2012 00:09:14 +0000 (UTC) Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50000298546.msg; Mon, 01 Oct 2012 01:08:29 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Mon, 01 Oct 2012 01:08:29 +0100 (not processed: message from trusted or authenticated source) X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=16217704b1=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <206FFD6BAB774B1196F4434F2661438B@multiplay.co.uk> From: "Steven Hartland" To: Date: Mon, 1 Oct 2012 01:08:35 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-hackers@freebsd.org Subject: Looking for testers / feedback for ZFS recieve properties options X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2012 00:09:18 -0000 We encountered a problem receiving a full ZFS stream from a disk we had backed up. The problem was the receive was aborting due to quota being exceeded so I did some digging around and found that Oracle ZFS now has -x and -o options as documented here:- http://docs.oracle.com/cd/E23824_01/html/821-1462/zfs-1m.html Seems this has been raised as a feature request upstream: https://www.illumos.org/issues/2745 Anyway being stuck with a backup we couldn't restore I had a play a implementing these options and have a prototype up and running, which I'd like feedback on. This patch also adds a -l option which allows the streams to be limited to those specified. Another option which I think would be useful and seemed relatively painless to add. The initial version of the patch which is based off 8.3-RELEASE can be found here: http://blog.multiplay.co.uk/dropzone/freebsd/zfs-recv-properties.patch Any feedback appreciated Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 1 16:01:57 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DA465106564A for ; Mon, 1 Oct 2012 16:01:57 +0000 (UTC) (envelope-from mark@exonetric.com) Received: from relay.exonetric.net (relay0.exonetric.net [178.250.72.161]) by mx1.freebsd.org (Postfix) with ESMTP id A20088FC0A for ; Mon, 1 Oct 2012 16:01:56 +0000 (UTC) Received: from markimac.fairfx.local (unknown [62.244.179.74]) by relay.exonetric.net (Postfix) with ESMTPSA id 1906D2C70B for ; Mon, 1 Oct 2012 16:56:18 +0100 (BST) From: Mark Blackman Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: Date: Mon, 1 Oct 2012 16:56:17 +0100 To: FreeBSD Hackers Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) X-Mailer: Apple Mail (2.1498) Subject: MDB ("Memory-Mapped Database"), a read-optimized database library X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2012 16:01:58 -0000 Hi, I wonder if this is an interesting library for FreeBSD for some userland applications. Perhaps an embedded app running on very low end hardware might benefit. Interesting use of an old idea in any case. http://www.openldap.org/pub/hyc/mdm-slides.pdf - Mark From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 1 16:51:10 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B6AAC106564A for ; Mon, 1 Oct 2012 16:51:10 +0000 (UTC) (envelope-from bfalk_bsd@brandonfa.lk) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 671948FC0C for ; Mon, 1 Oct 2012 16:51:10 +0000 (UTC) Received: by yenl8 with SMTP id l8so1124801yen.13 for ; Mon, 01 Oct 2012 09:51:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=brandonfa.lk; s=google; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=2q1+juwuIIRi3NdO3BX1r7zY+0eLTwxtE8+OmzN06EQ=; b=rbbpYQLZoecqVXjLjoLBHgypb3GjqzSC1nQkFcpZ4s7jdA7Vhg/tYT606Rc+J/29a2 n2gJEpNR6KTs87vOCIONKdONeZ4sNBt80WsAE4avB6dRe2KzsSq//c5KBMLQwUvF8EbR kcGo37QOHj4AURWaebrcjKgU9cjofTMpniR0k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding:x-gm-message-state; bh=2q1+juwuIIRi3NdO3BX1r7zY+0eLTwxtE8+OmzN06EQ=; b=XHCwRMEbZ+ehJhgOHlLvjP+6uMWp4nsRmzoUtgQLzr+tIfyMWrLZTILug8znu9Q4c0 FjtCctGuOxm4i1dewd7EZfMXHzpVUzg2f5/2RDl2IgeF12qq6XjK7od0MQpVIOZaxumb zxpfs5fTiptTkcrFCGg3yWDjyZkYHXZx1uquyZ90DQSp4UPbVQJVN5em4UipMGCXUbrT zktpk7t+TeF/RbOgI/SFBfOEqkM0jbmlcheIhmdQzBw+Fr6G3YkdrhSQZ54p8F+Xqph+ mJupvySUPLQdltQNRCok19fzhB7LDSaMjnuA/rT6RimwgqUg5SMaTGk7KcoOGSDtVN+C nD1A== Received: by 10.236.143.4 with SMTP id k4mr16085271yhj.111.1349110269533; Mon, 01 Oct 2012 09:51:09 -0700 (PDT) Received: from [10.1.10.29] (173-163-205-222-Richmond.hfc.comcastbusiness.net. [173.163.205.222]) by mx.google.com with ESMTPS id n20sm18171620ano.1.2012.10.01.09.51.08 (version=SSLv3 cipher=OTHER); Mon, 01 Oct 2012 09:51:08 -0700 (PDT) Message-ID: <5069C9FC.6020400@brandonfa.lk> Date: Mon, 01 Oct 2012 12:51:08 -0400 From: Brandon Falk User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQmp0bMm26UODdNZkQRNaGjp1LI4wfsu0IIRIkCswidI8a2MqkI8IWHvBdEbJoqfxfZGvaGJ Subject: SMP Version of tar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2012 16:51:10 -0000 I would be willing to work on a SMP version of tar (initially just gzip or something). I don't have the best experience in compression, and how to multi-thread it, but I think I would be able to learn and help out. Note: I would like to make this for *BSD under the BSD license. I am aware that there are already tools to do this (under GPL), but I would really like to see this existent in the FreeBSD base. Anyone interested? -Brandon From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 1 17:39:24 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AED6D1065674 for ; Mon, 1 Oct 2012 17:39:24 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-da0-f54.google.com (mail-da0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3D51A8FC15 for ; Mon, 1 Oct 2012 17:39:24 +0000 (UTC) Received: by dadz9 with SMTP id z9so1774397dad.13 for ; Mon, 01 Oct 2012 10:39:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=FPmVw5jEvjGI71F1qafCbE7anAVjHT+fz4zPfxjUod0=; b=UI2XQr75s0YNiGmfp0cVx6TEUBYMfeppgFyXf8H5/AWhiThyjbxyfzrQxJNq8MGdHT XVo/IXD50XLu3j8tVs3i77ru9A2/Cq34mMa9YdRuztALKQnpd3TAbOAC6SE6pdGF7Xf2 FpCNf0YKwChA/lLZhIa6RMP5vxN7i3IXF6lJw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-gm-message-state; bh=FPmVw5jEvjGI71F1qafCbE7anAVjHT+fz4zPfxjUod0=; b=eHOZMFQL34HqAi+soxvCiQ+2QgwPAGFx5CG8AIN/nJYgLfHA6f5pQHF/Nh7b8drmKy IZEFQz1G0sexfkyw1uxfhoeDnsr++keLlhOUDrJ6Y2RqZi4ZG2g1XOlhgqH4LmLsebgf 2rX6EWGO8nieSFkd50osbfnZ8Nofosw3QK+VsA/vWn1zPhm2XU+37AsDfHF2k/g9RGmu 3Vp3KNYmhPUQPHzcApUvJsKdJA3SEkAiNaSj6oigRnYZjsT/gzo6GvXAbXhoKo1Qwvjh s2pc9e5Zpwm94pxblb9yIy5cLY65AXYRQdPX/2ead8RzWaISPxIQkw6PDju7djpMZXn3 /JeQ== Received: by 10.66.77.40 with SMTP id p8mr38135566paw.78.1349113163888; Mon, 01 Oct 2012 10:39:23 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.190.164 with HTTP; Mon, 1 Oct 2012 10:38:53 -0700 (PDT) In-Reply-To: <5069C9FC.6020400@brandonfa.lk> References: <5069C9FC.6020400@brandonfa.lk> From: Eitan Adler Date: Mon, 1 Oct 2012 13:38:53 -0400 Message-ID: To: Brandon Falk Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQm4bCkAr77pmNwCgSVojY9J25FpJsg3QP+QVmc9JFyk7F1hJOWCWbgIXndkOoezhuNVFmvi Cc: freebsd-hackers@freebsd.org, Tim Kientzle Subject: Re: SMP Version of tar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2012 17:39:24 -0000 On 1 October 2012 12:51, Brandon Falk wrote: > I would be willing to work on a SMP version of tar (initially just gzip or > something). > > I don't have the best experience in compression, and how to multi-thread it, > but I think I would be able to learn and help out. > > Note: I would like to make this for *BSD under the BSD license. I am aware > that there are already tools to do this (under GPL), but I would really like > to see this existent in the FreeBSD base. > > Anyone interested? Tim Kientzle (cced) is the person to talk with. Try to keep questions on the mailing list with him CCed so we all learn. :) -- Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 1 18:01:05 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 061E31065674 for ; Mon, 1 Oct 2012 18:01:05 +0000 (UTC) (envelope-from jguojun@sbcglobal.net) Received: from nm27-vm3.bullet.mail.ne1.yahoo.com (nm27-vm3.bullet.mail.ne1.yahoo.com [98.138.91.157]) by mx1.freebsd.org (Postfix) with SMTP id AD6C68FC08 for ; Mon, 1 Oct 2012 18:01:04 +0000 (UTC) Received: from [98.138.226.178] by nm27.bullet.mail.ne1.yahoo.com with NNFMP; 01 Oct 2012 18:00:58 -0000 Received: from [66.94.237.119] by tm13.bullet.mail.ne1.yahoo.com with NNFMP; 01 Oct 2012 18:00:57 -0000 Received: from [127.0.0.1] by omp1024.access.mail.mud.yahoo.com with NNFMP; 01 Oct 2012 18:00:57 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 833630.96122.bm@omp1024.access.mail.mud.yahoo.com Received: (qmail 77574 invoked by uid 60001); 1 Oct 2012 18:00:57 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sbcglobal.net; s=s1024; t=1349114457; bh=BEm3yVxG4oKoIhpJkzFBtZf1kPmnqD01gaDswHKtmPc=; h=X-YMail-OSG:Received:X-Mailer:References:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=KwSqbjK/Eu0RZIFIObXtWf0TyU6W6uElFczN20zPdLz2xvy9oWFGpX9cqb+lPkZ4Y3YG1uNzWaOY1jv3J0ZkDRhES+X3S0xYcSyoF3XZkr2IK/Q60M75THSe8uLQ5pyrr7ipe8ZscI8Z7G+8XYqCxgZqSguAb0KQwavRQYN9kuA= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=sbcglobal.net; h=X-YMail-OSG:Received:X-Mailer:References:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=I4lbmWdkrtAazU6jxa0iJ6RNK3mlILaXrKOfrzW9Bm+pzbPa4olIgJtAslX99k6zD2VbJPANjqQXqEIw0hULMjakL3DLILB4m7pIGKoYaCK9DropFWNwbE7P093TuWOtqohLoV54ZWmLyOjKI8j+Zft0tCQ6sV0+qVfEcFYHQck=; X-YMail-OSG: aiyQ6R8VM1kUP.v.sntgPO1aFXgEwPVau9BHz3kowb.pGlR Ziq3gPMA6w1C3hAS3HA073J4Ljc1MpNkNvAGt_OxCs88qIt6maw7ibfrdlSG ZdrDwxcazvLyMiQV5k6gx.IxTHPcPN8oTjOapo6YwYdygbheNPRGrF4UjuXc toMqdk.Ys5W3mdmrQsTsCL1HWjZEVy2Q8pDwGoecj2i6caVy_G3cT4zAnv6c HIa1L_AJL03JUmL5d4l1kKv7wTshgB_v0MIqJcoUqlNTSNwnTIB8ziGUrAQ6 Y8lWVOy8ryhSszHTm9H2lLpIU2TixuWxJvE8Za12tdztL_qHTX7C3zSef0R4 2nqGqM8IhsyQZQZTDzOwJ8pcuN1kBnJn9fIaBFmIZ8SedeXgK8yvIWvqisLu cdstY9EUwAxCRngwcEeb9PJtoPQ.g6SFRUSpsrYnCrxD805kAiFuU0x4asDS 9NZ556xgTkEl4BoDRDZI4SBR2SXKi8kG3pcgWOk3HYwXmy9NGqmxnRyjUmL9 sWky. Received: from [75.37.1.111] by web180904.mail.ne1.yahoo.com via HTTP; Mon, 01 Oct 2012 11:00:57 PDT X-Mailer: YahooMailRC/708 YahooMailWebService/0.8.122.442 References: <1348984196.37002.YahooMailRC@web180906.mail.ne1.yahoo.com> <5068A688.4070100@delphij.net> Message-ID: <1349114457.65907.YahooMailRC@web180904.mail.ne1.yahoo.com> Date: Mon, 1 Oct 2012 11:00:57 -0700 (PDT) From: Jin Guojun To: d@delphij.net In-Reply-To: <5068A688.4070100@delphij.net> MIME-Version: 1.0 X-Mailman-Approved-At: Mon, 01 Oct 2012 18:34:17 +0000 Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: hackers@freebsd.org, questions@freebsd.org Subject: Re: system hangs during dump + compress > usb2-drive X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2012 18:01:05 -0000 ________________________________ From: Xin Li To: Jin Guojun Cc: questions@freebsd.org; hackers@freebsd.org Sent: Sun, September 30, 2012 1:07:40 PM Subject: Re: system hangs during dump + compress > usb2-drive On 9/29/12 10:49 PM, Jin Guojun wrote: > In FreeBSD 8.3 release (possibly in earlier release), dump a file > system has 2-3GB or more content can cause system hang in a > specific case (pipe to compression): > > dump FS-on-SATA-drive > usb-drive OK dump FS-on-SATA-drive > | anyCompress > sata-drive OK mv a-large-dump-file from > STAT drive to a USB drive OK dump small-FS-on-SATA-drive | > anyCompress > usb-drive OK small -- 1.8GB or less dump > large-FS-on-SATA-drive | anyCompress > usb-drive hang > content is 3GB or larger (did not try around 2GB yet) > > When system hangs, no sub system, such video, network, etc, will > function. Typically, the unfinished compressed dump file is around > 1.5-2.7GB, so guessing dumped file content is close to or over 2GB > when failure occurred. > > Has anyone encountered the same problem? > > Because this usually takes a few hours to occur, this is hard to > watch how/when it happens. Is any way to debug or determine what > status the system is? For starters I'd use a different console for doing procstat -kk -a and see what the system is doing. (Perhaps also top) I *think* that if it's just hanging for some time, it's probably because the system is trying to take a snapshot? It takes time on UFS when creating and removing the snapshot. Just a guess... Cheers, --------------- Not sure how to use a different console. No tty is functioning (neither ttyv? nor over network). You are right on a different case -- mount /dev/da0s4d /mnt # mount a usb drive cd /mnt ssh remote-liux-host "tar -cf - 8GB_FS" | tar -xf - In this case, doing ls -l /mnt or df will hangs, but system is still alive. The network is 45Mbps. I have no idea how long it took the tar to finish since machine is 60 km away. When I left there last Friday, only 400MB was done in one hour. I will get the processing time tomorrow. The problem we can see now is that tar (probably the pipe) process only finish with 4GB. # df Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/ad4s3a 1012974 355348 576590 38% / devfs 1 1 0 100% /dev ... /dev/da0s4d 1027486774 4198246 941089588 0% /mnt So, I suspect this is a pipe problem, not a compress issue. From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 1 19:43:02 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4781D106564A for ; Mon, 1 Oct 2012 19:43:02 +0000 (UTC) (envelope-from geoffrey.levand@mail.ru) Received: from fallback1.mail.ru (fallback1.mail.ru [94.100.176.18]) by mx1.freebsd.org (Postfix) with ESMTP id A9D9D8FC0A for ; Mon, 1 Oct 2012 19:43:01 +0000 (UTC) Received: from f243.mail.ru (f243.mail.ru [94.100.178.218]) by fallback1.mail.ru (mPOP.Fallback_MX) with ESMTP id 9614EA6F03B for ; Mon, 1 Oct 2012 21:42:55 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ru; s=mail; h=Content-Type:Message-ID:Reply-To:In-Reply-To:References:Date:Mime-Version:Subject:Cc:To:From; bh=O//hAdEb+O6QjPlNyULJEHhcdsJtxLwI18/rqVOG6ZI=; b=QGX7zKSluohp7XaSX8Ts70TVqPPfDP7eDtB+5oV8SlfPeZUsOHLPPkaNsx7shdKfUxQcX+Vi6w5BihYHC4ZwsRQNpwhdqsGXR4loDXJuiRelHWDVWAT2utnNNOOoXlgg; Received: from mail by f243.mail.ru with local (envelope-from ) id 1TIk11-0008QJ-Lt; Mon, 01 Oct 2012 21:42:47 +0400 Received: from [109.193.221.117] by e.mail.ru with HTTP; Mon, 01 Oct 2012 21:42:47 +0400 From: =?UTF-8?B?Z2VvZmZyZXkgbGV2YW5k?= To: =?UTF-8?B?SGFucyBQZXR0ZXIgU2VsYXNreQ==?= Mime-Version: 1.0 X-Mailer: mPOP Web-Mail 2.19 X-Originating-IP: [109.193.221.117] Date: Mon, 01 Oct 2012 21:42:47 +0400 References: <1348410653.373800982@f180.mail.ru> <1348910721.385146900@f89.mail.ru> <201209302133.31641.hselasky@c2i.net> In-Reply-To: <201209302133.31641.hselasky@c2i.net> X-Priority: Message-ID: <1349113367.263372922@f243.mail.ru> X-Spam: Not detected X-Mras: Ok Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re[2]: How to claim only some of USB interfaces of a composite USB device X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: =?UTF-8?B?Z2VvZmZyZXkgbGV2YW5k?= List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2012 19:43:02 -0000 ClRlc3RlZCB0aGUgcGF0Y2ggd2l0aCBGcmVlQlNEIDkuMS4KSXQgd29ya3MuClRoYW5rcy4KClN1 biwgMzAgU2VwIDIwMTIgMjE6MzM6MzEgKzAyMDAg0L7RgiBIYW5zIFBldHRlciBTZWxhc2t5IDxo c2VsYXNreUBjMmkubmV0PjoKPgkKPgo+CgkKCQo+CgkJCgkJCgkJCQo+T24gU2F0dXJkYXkgMjkg U2VwdGVtYmVyIDIwMTIgMTE6MjU6MjEgZ2VvZmZyZXkgbGV2YW5kIHdyb3RlOgo+Cj4gZ2VvZmZy ZXkgbGV2YW5kCj4KPgpDYW4geW91IHZlcmlmeSB0aGlzIHBhdGNoOgo+Cj5odHRwOi8vc3ZuLmZy ZWVic2Qub3JnL2NoYW5nZXNldC9iYXNlLzI0MTA3OAo+Cj4KLS1IUFMKPgpfX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwo+ZnJlZWJzZC1oYWNrZXJzQGZyZWVi c2Qub3JnIG1haWxpbmcgbGlzdAo+aHR0cDovL2xpc3RzLmZyZWVic2Qub3JnL21haWxtYW4vbGlz dGluZm8vZnJlZWJzZC1oYWNrZXJzCj4KVG8gdW5zdWJzY3JpYmUsIHNlbmQgYW55IG1haWwgdG8g ImZyZWVic2QtaGFja2Vycy11bnN1YnNjcmliZUBmcmVlYnNkLm9yZyIKPgoJCQkKCQkKCQkKCQoK CQo= From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 1 20:00:41 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D7F4B106564A; Mon, 1 Oct 2012 20:00:41 +0000 (UTC) (envelope-from guy.helmer@gmail.com) Received: from mail-vc0-f186.google.com (mail-vc0-f186.google.com [209.85.220.186]) by mx1.freebsd.org (Postfix) with ESMTP id 65EAB8FC0A; Mon, 1 Oct 2012 20:00:41 +0000 (UTC) Received: by vcbfy7 with SMTP id fy7so5085663vcb.13 for ; Mon, 01 Oct 2012 13:00:40 -0700 (PDT) Received: by 10.236.201.134 with SMTP id b6mr2228394yho.15.1349121640824; Mon, 01 Oct 2012 13:00:40 -0700 (PDT) Path: glegroupsg2000goo.googlegroups.com!not-for-mail Newsgroups: fa.freebsd.hackers Date: Mon, 1 Oct 2012 13:00:40 -0700 (PDT) In-Reply-To: Complaints-To: groups-abuse@google.com Injection-Info: glegroupsg2000goo.googlegroups.com; posting-host=216.81.189.9; posting-account=UWBIDAoAAADr2Rwohhx_YBn6X-E8F7Gr NNTP-Posting-Host: 216.81.189.9 References: User-Agent: G2/1.0 X-Google-Web-Client: true X-Google-IP: 216.81.189.9 MIME-Version: 1.0 Message-ID: <452f3689-b2ad-43e2-8835-8691b25f75c9@googlegroups.com> From: guy.helmer@gmail.com To: fa.freebsd.hackers@googlegroups.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Mon, 01 Oct 2012 20:38:55 +0000 Cc: freebsd-hackers@freebsd.org, freebsd-questions@freebsd.org Subject: Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2012 20:00:42 -0000 On Wednesday, June 6, 2012 8:36:04 PM UTC-5, Mark Felder wrote: > Hi guys I'm excitedly posting this from my phone. Good news for you guys,= bad news for us -- we were building HA storage on vmware for a client and = can now replicate the crash on demand. I'll be posting details when I get h= ome to my PC tonight, but this hopefully is enough to replicate the crash f= or any curious followers: >=20 >=20 >=20 > ESXi 5 >=20 > 9 or 9-STABLE >=20 > HAST=20 >=20 > 1 cpu is fine >=20 > 1GB of ram >=20 > UFS SUJ on HAST device >=20 > No special loader.conf, sysctl, etc >=20 > No need for VMWare tools >=20 > Run Bonnie++ on the HAST device >=20 >=20 >=20 > We can get the crash to happen on the first run of bonnie++ right now. I'= ll post the exact specs and precise command run in the PR. We found an old = post from 2004 when we looked up the process state obtained from CTRL+T -- = flswai -- which describes the symptoms nearly perfectly. >=20 >=20 >=20 > http://unix.derkeiler.com/Mailing-Lists/FreeBSD/stable/2004-02/0250.html= =20 >=20 >=20 >=20 > Hopefully this gets us closer to a fix... Is this a crash or a hang? Over the past couple of weeks, I've been working= with a FreeBSD 9.1RC1 system under VMware ESXi 5.0 with a 64GB UFS root FS= and 2TB ZFS filesystem mounted via a virtual LSI SAS interface. Sometimes = during heavy I/O load (rsync from other servers) on the ZFS FS, this shows = up in /var/log/messages: Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 5 ee= 60 16 0 1 0 0=20 Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI Status E= rror Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): Retrying command Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 3 ef= 42 51 0 1 0 0=20 Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI Status E= rror Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): Retrying command Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 3 ef= 64 51 0 1 0 0=20 Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI Status E= rror Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): Retrying command Sep 21 02:18:49 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 3 ef= 66 51 0 1 0 0=20 Sep 21 02:18:49 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI Status E= rror Sep 21 02:18:49 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy ... Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 41 f= 3 94 99 0 1 0 0=20 Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI Status E= rror Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): Retrying command These have been happening roughly every other day. mpt0 and em0 were sharing int 18, so today I put=20 hint.mpt.0.msi_enable=3D"1" into /boot/devices.hints and rebooted; now mpt0 is using int 256. I'll see = if it helps. Guy From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 1 20:44:27 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55E181065670; Mon, 1 Oct 2012 20:44:27 +0000 (UTC) (envelope-from feld@feld.me) Received: from feld.me (unknown [IPv6:2607:f4e0:100:300::2]) by mx1.freebsd.org (Postfix) with ESMTP id 16FC78FC08; Mon, 1 Oct 2012 20:44:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=feld.me; s=blargle; h=In-Reply-To:Message-Id:From:Mime-Version:Date:References:Subject:Cc:To:Content-Type; bh=Xc4ePcvv042pfG8iZcGFuMeNitkfJx2rwrUlgtybuw4=; b=QzupTyA3Lram0BBLgw53f4F/lnP1Mktu8WeTHGAWiKDHc4uBGiiK95pq4Lw2W2xQ1XARY7BX8Gifr73aeCr2G0TaLcCnF/ivcbHIjvlu6ypnzV+VEVpg+GGsuj80EPGn; Received: from localhost ([127.0.0.1] helo=mwi1.coffeenet.org) by feld.me with esmtp (Exim 4.80 (FreeBSD)) (envelope-from ) id 1TImqg-0006C5-7q; Mon, 01 Oct 2012 15:44:21 -0500 Received: from feld@feld.me by mwi1.coffeenet.org (Archiveopteryx 3.1.4) with esmtpa id 1349124252-3100-3099/5/150; Mon, 1 Oct 2012 20:44:12 +0000 Content-Type: text/plain; charset=iso-8859-15; format=flowed; delsp=yes To: fa.freebsd.hackers@googlegroups.com, guy.helmer@gmail.com References: <452f3689-b2ad-43e2-8835-8691b25f75c9@googlegroups.com> Date: Mon, 1 Oct 2012 15:44:37 -0500 Mime-Version: 1.0 From: Mark Felder Message-Id: In-Reply-To: <452f3689-b2ad-43e2-8835-8691b25f75c9@googlegroups.com> User-Agent: Opera Mail/12.02 (Win32) X-SA-Report: ALL_TRUSTED=-1, KHOP_THREADED=-0.5 X-SA-Score: -1.5 Cc: freebsd-hackers@freebsd.org, freebsd-questions@FreeBSD.org Subject: Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2012 20:44:27 -0000 On Mon, 01 Oct 2012 15:00:40 -0500, wrote: > > Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 5 > ee 60 16 0 1 0 0 > Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI > Status Error > Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy > Sep 21 02:14:55 backups kernel: (da1:mpt0:0:1:0): Retrying command > Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 3 > ef 42 51 0 1 0 0 > Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI > Status Error > Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy > Sep 21 02:18:44 backups kernel: (da1:mpt0:0:1:0): Retrying command > Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 3 > ef 64 51 0 1 0 0 > Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI > Status Error > Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy > Sep 21 02:18:48 backups kernel: (da1:mpt0:0:1:0): Retrying command > Sep 21 02:18:49 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 3 > ef 66 51 0 1 0 0 > Sep 21 02:18:49 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI > Status Error > Sep 21 02:18:49 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy > ... > Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): WRITE(10). CDB: 2a 0 > 41 f3 94 99 0 1 0 0 > Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): CAM status: SCSI > Status Error > Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): SCSI status: Busy > Sep 21 05:06:18 backups kernel: (da1:mpt0:0:1:0): Retrying command Sometimes you'll see this before a crash, but not every time. From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 1 22:31:51 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 755E4106566C; Mon, 1 Oct 2012 22:31:51 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from exprod7og121.obsmtp.com (exprod7og121.obsmtp.com [64.18.2.20]) by mx1.freebsd.org (Postfix) with ESMTP id 416EE8FC14; Mon, 1 Oct 2012 22:31:49 +0000 (UTC) Received: from P-EMHUB03-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob121.postini.com ([64.18.6.12]) with SMTP ID DSNKUGoZ1ekWA20MmS+tkbQfzOTwPUxM0ev4@postini.com; Mon, 01 Oct 2012 15:31:51 PDT Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB03-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Mon, 1 Oct 2012 15:31:02 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id q91MV1h39000; Mon, 1 Oct 2012 15:31:01 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id E7D0D58093; Mon, 1 Oct 2012 15:31:00 -0700 (PDT) To: Garrett Cooper In-Reply-To: References: Comments: In-reply-to: Marcel Moolenaar message dated "Mon, 01 Oct 2012 12:17:52 -0700." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Mon, 1 Oct 2012 15:31:00 -0700 Message-ID: <20121001223100.E7D0D58093@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-Mailman-Approved-At: Mon, 01 Oct 2012 22:53:27 +0000 Cc: freebsd-hackers@FreeBSD.org, freebsd-arch@FreeBSD.org, sjg@juniper.net Subject: Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2012 22:31:51 -0000 Hi Garrett, >> From: Garrett Cooper >> Subject: [CFT/RFC]: refactor bsd.prog.mk to understand multiple = >programs instead of a singular program >> Date: September 2, 2012 11:01:09 PM PDT >> To: freebsd-hackers@freebsd.org >> Cc: "freebsd-arch@FreeBSD.org Arch" >>=20 >> Hello, >> I've been a bit busy working on porting over ATF from NetBSD, and Thanks, we've been using ATF in Junos for a while and glad to see it being imported to FreeBSD. >> one of the pieces that's currently not available in FreeBSD that's >> available in NetBSD is the ability to understand and compile multiple >> programs. In order to do this I had to refactor bsd.prog.mk (a lot). A change like this to bsd.prog.mk can have considerable fallout. Eg. any makefile that tweaks OBJS is suddenly out of luck. Not to mention the fact that bsd.prog.mk goes from being relatively simple, to unspeakably hard to read, and all for rather limited return. Apart from ATF, is there any huge demand for building multiple progs in the same directory? FWIW we use progs.mk (as bsd.progs.mk) from ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz It isn't ideal, but it certainly avoids a lot of churn and complexity for what is essentially a corner case. We have an atf.test.mk which leverages that. It also handles automatically running the tests if building for the host. This allows us to fail the build if any unit-tests do not pass. Note, atf.test.mk is loosely derrived from NetBSD's bsd.tests.mk, but named for what it is (ATF specific tests) since we wanted the flexibility to have more than one test framework. --sjg From owner-freebsd-hackers@FreeBSD.ORG Mon Oct 1 23:14:51 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A7EAB1065672; Mon, 1 Oct 2012 23:14:51 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6E9528FC0A; Mon, 1 Oct 2012 23:14:51 +0000 (UTC) Received: by padbi1 with SMTP id bi1so5329518pad.13 for ; Mon, 01 Oct 2012 16:14:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=CBDb5KN/i0qH0pY1Y4rPEUQWiLer88/3s3eZFuBshHY=; b=buB2kkAXTkStesPl5TJcWyHSnQQn8mEPDLuP6iB61E5WkQyrzmdN0M3QrTrELNiURx y3lysYxRNbRr2YDrz7XEB6bXLGUbGnhhBOqWzpulaXXE/k7s+kSZQ8z7xaNGK3qLBJe2 onUHrrgO3yX/lvItz1G3X/jtrhwxyvgjbQVUndZDMRWNU/7iFZ4QF8u1vgnlO5ZnMTO7 wXWY8AqA9BNWg99NFh3HihlREMVdUH7AW6zx1l/UUJvtnnKnFsIqWiXWY2/k1ooeqdA6 jxF+sfMcW4kE4bZJ8nV/LwFTWGp3rz6jacKCkWqqaVw4WS/iMVwGE/N2HGz4Fwq1bUdd ge4g== Received: by 10.66.88.233 with SMTP id bj9mr39656786pab.72.1349133290901; Mon, 01 Oct 2012 16:14:50 -0700 (PDT) Received: from fuji-wireless.local (c-24-19-191-56.hsd1.wa.comcast.net. [24.19.191.56]) by mx.google.com with ESMTPS id py9sm10951798pbb.20.2012.10.01.16.14.48 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 01 Oct 2012 16:14:49 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=windows-1252 From: Garrett Cooper In-Reply-To: <20121001223100.E7D0D58093@chaos.jnpr.net> Date: Mon, 1 Oct 2012 16:15:16 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20121001223100.E7D0D58093@chaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1278) Cc: freebsd-hackers@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2012 23:14:51 -0000 Hi Simon! On Oct 1, 2012, at 3:31 PM, Simon J. Gerraty wrote: > Hi Garrett, >=20 >>> From: Garrett Cooper >>> Subject: [CFT/RFC]: refactor bsd.prog.mk to understand multiple =3D >> programs instead of a singular program >>> Date: September 2, 2012 11:01:09 PM PDT >>> To: freebsd-hackers@freebsd.org >>> Cc: "freebsd-arch@FreeBSD.org Arch" >>> =3D20 >>> Hello, >>> I've been a bit busy working on porting over ATF from NetBSD, and >=20 > Thanks, we've been using ATF in Junos for a while and glad to see it=20= > being imported to FreeBSD. Thanks (and I appreciate the help marcel@ has given by taking the first = step in importing it). >>> one of the pieces that's currently not available in FreeBSD that's >>> available in NetBSD is the ability to understand and compile = multiple >>> programs. In order to do this I had to refactor bsd.prog.mk (a lot). >=20 > A change like this to bsd.prog.mk can have considerable fallout. > Eg. any makefile that tweaks OBJS is suddenly out of luck. Well=85 any Makefiles attempting to be "clever" (e.g. crunchgen) are out = of luck and would need to change. Most [include] Makefiles will see = virtually no change. > Not to mention the fact that bsd.prog.mk goes from being relatively > simple, to unspeakably hard to read, and all for rather limited = return.=20 I sort of tried to match bsd.prog.mk in NetBSD in the beginning, but = things diverged from there=85 I'm sure things could be made more = readable so any comments would be much appreciated :)! > Apart from ATF, is there any huge demand for building multiple progs = in > the same directory? Well, I noticed that there are a couple ugly messes in the gnu/... = directory that attempt to work around the fact that bsd.prog.mk doesn't = support more than one program by being tricky and emulating bsd.prog.mk = instead of solving the generic problem (and once I got over that = compatibility issue I stopped tracking this class of consumer). Most = consumers don't care (they split up programs into different directories = or use hardlink hacks/basename detection to differentiate one program = functionally from another). Getting back to the idea at hand, the enhancement goal was to get the = testcases to live [and optionally be built/installed] with the source = code to avoid bitrot; I know this isn't the current NetBSD design, but = this is what was requested be done by gnn, marcel, and mdf, and I agree. = It order to do this, I need to be able to build multiple programs so = things are as transparent. On the flipside, bsd.prog[s].mk can live on = its own, be pulled in automatically by bsd.test.mk, and that would be = it. This encourages code duplication though and bugs can persist in = either Makefile, when I'd rather work out all the kinks in whatever = succeeds the legacy bsd.prog.mk file. > FWIW we use progs.mk (as bsd.progs.mk) from > ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz=20 > It isn't ideal, but it certainly avoids a lot of churn and complexity > for what is essentially a corner case. This requires pulling bmake into FreeBSD proper in order for things to = function the last I checked as bsd.prog.mk depends upon bmake directives = in order to function properly 100% of the time (and there are external = dependencies and assumptions one has to deal with when using = bsd.prog.mk, etc from NetBSD that made that a bit of a no-go without = refactoring/pulling in bits from NetBSD). I could be wrong though, so = please let me know if I am :). Ideally however, I would like doing this compared to running a custom = build infrastructure, but (as you probably know) this requires = rototilling the current FreeBSD build system to a large degree = (definitely not impossible=85 just needs time and effort). > We have an atf.test.mk which leverages that. > It also handles automatically running the tests if building for the > host. This allows us to fail the build if any unit-tests do not pass. Hmm=85 that wasn't really the end-goal of bsd.test.mk based on my = reading, but I'm sure Julio would be interested in it. I need to do = adjusting in order to allow automatically executing testcases compatible = to the host architecture, but that isn't hard to do (no more than a = couple hours work). > Note, atf.test.mk is loosely derrived from NetBSD's bsd.tests.mk, but > named for what it is (ATF specific tests) since we wanted the > flexibility to have more than one test framework. bsd.test.mk in NetBSD isn't completely test-agnostic, but it isn't = entirely ATF centric either. I'm trying not to stray too far from NetBSD = in this area because there really isn't any reason to do so. FWIW I'd = like to see other test frameworks (lua, unittest//nose, etc) just snap = into bsd.test.mk as easily as possible as it would make some groups = lives easier, but that requires a bit more thought for another day. Thanks for the feedback! -Garrett= From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 2 00:02:12 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3FE6C1065674; Tue, 2 Oct 2012 00:02:12 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from exprod7og110.obsmtp.com (exprod7og110.obsmtp.com [64.18.2.173]) by mx1.freebsd.org (Postfix) with ESMTP id 978518FC0C; Tue, 2 Oct 2012 00:02:10 +0000 (UTC) Received: from P-EMHUB03-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob110.postini.com ([64.18.6.12]) with SMTP ID DSNKUGovAWxyWzC4IQvD1sZjrFmUuVrQOwO3@postini.com; Mon, 01 Oct 2012 17:02:11 PDT Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB03-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Mon, 1 Oct 2012 17:00:32 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id q9200Uh40887; Mon, 1 Oct 2012 17:00:31 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 54CEE58093; Mon, 1 Oct 2012 17:00:30 -0700 (PDT) To: Garrett Cooper In-Reply-To: References: <20121001223100.E7D0D58093@chaos.jnpr.net> Comments: In-reply-to: Garrett Cooper message dated "Mon, 01 Oct 2012 16:15:16 -0700." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Mon, 1 Oct 2012 17:00:30 -0700 Message-ID: <20121002000030.54CEE58093@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-Mailman-Approved-At: Tue, 02 Oct 2012 02:24:44 +0000 Cc: freebsd-hackers@FreeBSD.org, freebsd-arch@FreeBSD.org, sjg@juniper.net Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 00:02:12 -0000 >> Not to mention the fact that bsd.prog.mk goes from being relatively >> simple, to unspeakably hard to read, and all for rather limited = >return. This btw I think is the more important issue. I was looking at bsd.prog.mk in netbsd the other day. It has no business being that complex. >Getting back to the idea at hand, the enhancement goal was to get the = >testcases to live [and optionally be built/installed] with the source = >code to avoid bitrot; I know this isn't the current NetBSD design, but = >this is what was requested be done by gnn, marcel, and mdf, and I agree. = I agree, that's what we do too. >It order to do this, I need to be able to build multiple programs so = >things are as transparent. On the flipside, bsd.prog[s].mk can live on = We put the test cases in a subdir of the lib/prog This has multiple benefits, and eliminates any impact on the normal build of said libs/progs. Also a number of our teams find it necessary to create mock classes etc to adequately test their libs. Keeping all this in a tests/ subdir makes it easy, to keep things neat & up to date, and ensure that the tests pass. Trying to do all that in the same dir as the lib would be ugly. >> FWIW we use progs.mk (as bsd.progs.mk) from >> ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz=20 >> It isn't ideal, but it certainly avoids a lot of churn and complexity >> for what is essentially a corner case. > >This requires pulling bmake into FreeBSD proper in order for things to = >function the last I checked as bsd.prog.mk depends upon bmake directives = This is already happening ;-) >Ideally however, I would like doing this compared to running a custom = >build infrastructure, but (as you probably know) this requires = >rototilling the current FreeBSD build system to a large degree = Actually building FreeBSD-current using bmake requires only modest changes. >> We have an atf.test.mk which leverages that. >> It also handles automatically running the tests if building for the >> host. This allows us to fail the build if any unit-tests do not pass. > >Hmm=85 that wasn't really the end-goal of bsd.test.mk based on my = >reading, I know, but it is a very useful thing to be able to do. You can add knobs to controll such things of course. >bsd.test.mk in NetBSD isn't completely test-agnostic, but it isn't = >entirely ATF centric either. I'm trying not to stray too far from NetBSD = >in this area because there really isn't any reason to do so. FWIW I'd = Sure - we imported ATF directly from NetBSD to minimize the churn and not had to change much at all. bsd.test.mk was a point worth diverging on though. >like to see other test frameworks (lua, unittest//nose, etc) just snap = >into bsd.test.mk as easily as possible as it would make some groups = Yes, but making one test.mk handle potentially lots of frameworks will get rather ugly quite quickly. Better to add per-framework .mk files, and perhaps have them include a bsd.test.mk which only handles high level logic rather than the details of the frameworks. That's laregly why we didn't use bsd.test.mk --sjg From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 2 04:21:20 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A9C9106566B; Tue, 2 Oct 2012 04:21:20 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 04D8F8FC0A; Tue, 2 Oct 2012 04:21:19 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.5/8.14.5) with ESMTP id q924LIH4053924; Tue, 2 Oct 2012 00:21:18 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.5/8.14.4/Submit) id q924LIUG053921; Tue, 2 Oct 2012 00:21:18 -0400 (EDT) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <20586.27582.478147.838896@hergotha.csail.mit.edu> Date: Tue, 2 Oct 2012 00:21:18 -0400 From: Garrett Wollman To: hackers@freebsd.org X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (hergotha.csail.mit.edu [127.0.0.1]); Tue, 02 Oct 2012 00:21:18 -0400 (EDT) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: rmacklem@freebsd.org, jkoshy@freebsd.org Subject: NFS server bottlenecks X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 04:21:20 -0000 I had an email conversation with Rick Macklem about six months ago about NFS server bottlenecks. I'm now in a position to observe my large-scale NFS server under an actual production load, so I thought I would update folks on what it looks like. This is a 9.1 prerelease kernel (I hope 9.1 will be released soon as I have four moe of these servers to deploy!). When under nearly 100% load on an 8-core (16-thread) Quanta QSSC-S99Q storage server, with a 10G network interface, pmcstat tells me this: PMC: [INST_RETIRED.ANY_P] Samples: 2727105 (100.0%) , 27 unresolved Key: q => exiting... %SAMP IMAGE FUNCTION CALLERS 29.3 kernel _mtx_lock_sleep nfsrvd_updatecache:10.0 nfsrvd_getcache:7.4 ... 9.5 kernel cpu_search_highest cpu_search_highest:8.1 sched_idletd:1.4 7.4 zfs.ko lzjb_decompress zio_decompress 4.3 kernel _mtx_lock_spin turnstile_trywait:2.2 pmclog_reserve:1.0 ... 4.0 zfs.ko fletcher_4_native zio_checksum_error:3.1 zio_checksum_compute:0.8 3.6 kernel cpu_search_lowest cpu_search_lowest 3.3 kernel nfsrc_trimcache nfsrvd_getcache:1.6 nfsrvd_updatecache:1.6 2.3 kernel ipfw_chk ipfw_check_hook 2.1 pmcstat _init 1.1 kernel _sx_xunlock 0.9 kernel _sx_xlock 0.9 kernel spinlock_exit This does seem to confirm my original impression that the NFS replay cache is quite expensive. Running a gprof(1) analysis on the same PMC data reveals a bit more detail (I've removed some uninteresting parts of the call graph): called/total parents index %time self descendents called+self name index called/total children 4881.00 2004642.70 932627/932627 svc_run_internal [2] [4] 45.1 4881.00 2004642.70 932627 nfssvc_program [4] 13199.00 504436.33 584319/584319 nfsrvd_updatecache [9] 23075.00 403396.18 468009/468009 nfsrvd_getcache [14] 1032.25 416249.44 2239/2284 svc_sendreply_mbuf [15] 6168.00 381770.44 11618/11618 nfsrvd_dorpc [24] 3526.87 86869.88 112478/112514 nfsrvd_sentcache [74] 890.00 50540.89 4252/4252 svc_getcred [101] 14876.60 32394.26 4177/24500 crfree [263] 11550.11 25150.73 3243/24500 free [102] 1348.88 15451.66 2716/16831 m_freem [59] 4066.61 216.81 1434/1456 svc_freereq [321] 2342.15 677.40 557/1459 malloc_type_freed [265] 59.14 1916.84 134/2941 crget [113] 1602.25 0.00 322/9682 bzero [105] 690.93 0.00 43/44 getmicrotime [571] 287.22 7.33 138/1205 prison_free [384] 233.61 0.00 60/798 PHYS_TO_VM_PAGE [358] 203.12 0.00 94/230 nfsrv_mallocmget_limit [632] 151.76 0.00 51/1723 pmap_kextract [309] 0.78 70.28 9/3281 _mtx_unlock_sleep [154] 19.22 16.88 38/400403 nfsrc_trimcache [26] 11.05 21.74 7/197 crsetgroups [532] 30.37 0.00 11/6592 critical_enter [190] 25.50 0.00 9/36 turnstile_chain_unlock [844] 24.86 0.00 3/7 nfsd_errmap [913] 12.36 8.57 8/2145 in_cksum_skip [298] 9.10 3.59 5/12455 mb_free_ext [140] 1.84 4.85 2/2202 VOP_UNLOCK_APV [269] ----------------------------------------------- 0.49 0.15 1/1129009 uhub_explore [1581] 0.49 0.15 1/1129009 tcp_output [10] 0.49 0.15 1/1129009 pmap_remove_all [1141] 0.49 0.15 1/1129009 vm_map_insert [236] 0.49 0.15 1/1129009 vnode_create_vobject [281] 0.49 0.15 1/1129009 biodone [351] 0.49 0.15 1/1129009 vm_object_madvise [670] 0.49 0.15 1/1129009 xpt_done [483] 0.49 0.15 1/1129009 vputx [80] 0.49 0.15 1/1129009 vm_map_delete [49] 0.49 0.15 1/1129009 vm_object_deallocate [356] 0.49 0.15 1/1129009 vm_page_unwire [338] 0.49 0.15 1/1129009 pmap_change_wiring [318] 0.98 0.31 2/1129009 getnewvnode [227] 0.98 0.31 2/1129009 pmap_clear_reference [1004] 0.98 0.31 2/1129009 usbd_do_request_flags [1282] 0.98 0.31 2/1129009 vm_object_collapse [587] 0.98 0.31 2/1129009 vm_object_page_remove [122] 1.48 0.46 3/1129009 mpt_pci_intr [487] 1.48 0.46 3/1129009 pmap_extract [355] 1.48 0.46 3/1129009 vm_fault_unwire [171] 1.97 0.62 4/1129009 vgonel [270] 1.97 0.62 4/1129009 vm_object_shadow [926] 1.97 0.62 4/1129009 zone_alloc_item [434] 2.46 0.77 5/1129009 vnlru_free [235] 2.46 0.77 5/1129009 insmntque1 [737] 2.95 0.93 6/1129009 zone_free_item [409] 3.94 1.24 8/1129009 pmap_enter_object [805] 4.92 1.55 10/1129009 vfs_ref [722] 4.92 1.55 10/1129009 pmap_clear_modify [814] 5.41 1.70 11/1129009 vn_isdisk [391] 5.90 1.86 12/1129009 rtalloc1_fib [158] 5.90 1.86 12/1129009 soreceive_generic [99] 6.40 2.01 13/1129009 pmap_remove_pages [403] 6.89 2.16 14/1129009 _mtx_lock_flags [565] 6.89 2.16 14/1129009 random_yarrow_read [542] 6.89 2.16 14/1129009 nfsrv_checkgetattr [1133] 7.38 2.32 15/1129009 svc_vc_recv [55] 7.38 2.32 15/1129009 ixgbe_rxeof [12] 8.85 2.78 18/1129009 _cv_timedwait_sig [859] 9.35 2.94 19/1129009 vref [242] 10.82 3.40 22/1129009 pmap_enter_quick [690] 11.31 3.56 23/1129009 tcp_do_segment [35] 13.28 4.18 27/1129009 sosend_generic [25] 15.74 4.95 32/1129009 g_io_schedule_down [76] 19.19 6.03 39/1129009 vn_start_write [400] 20.66 6.49 42/1129009 ip_output [32] 23.12 7.27 47/1129009 pmap_copy [467] 23.12 7.27 47/1129009 __mnt_vnode_next_all [520] 29.52 9.28 60/1129009 vm_page_free_toq [149] 30.50 9.59 62/1129009 vm_fault_hold [268] 36.90 11.60 75/1129009 g_io_request [172] 38.37 12.06 78/1129009 prison_hold [330] 38.86 12.22 79/1129009 taskqueue_member [423] 39.36 12.37 80/1129009 vm_page_deactivate [812] 39.85 12.53 81/1129009 vm_page_activate [790] 40.83 12.83 83/1129009 in_matroute [197] 45.75 14.38 93/1129009 vm_page_alloc [128] 48.70 15.31 99/1129009 prison_free [384] 66.90 21.03 136/1129009 vholdl [275] 73.30 23.04 149/1129009 vfs_unbusy [470] 82.65 25.98 168/1129009 key_addref [233] 85.60 26.91 174/1129009 vn_finished_write [407] 91.50 28.76 186/1129009 _key_freesp [286] 92.48 29.07 188/1129009 vfs_busy [288] 102.32 32.16 208/1129009 _vm_object_allocate [738] 107.24 33.71 218/1129009 taskqueue_enqueue [68] 112.16 35.26 228/1129009 xprt_active [267] 125.94 39.59 256/1129009 vm_pageq_remove [650] 128.89 40.51 262/1129009 svc_freereq [321] 128.89 40.51 262/1129009 pmap_remove [106] 153.49 48.25 312/1129009 vdropl [251] 154.96 48.71 315/1129009 vfs_busyfs [239] 180.05 56.60 366/1129009 pmap_enter [245] 182.02 57.22 370/1129009 svc_run_internal [2] 213.50 67.11 434/1129009 ixgbe_msix_que [11] 248.43 78.09 505/1129009 key_allocsp [221] 255.32 80.26 519/1129009 camisr [107] 373.87 117.52 760/1129009 ipmi_poll [635] 404.87 127.27 823/1129009 g_io_deliver [162] 440.78 138.56 896/1129009 g_io_schedule_up [137] 462.91 145.51 941/1129009 taskqueue_run_locked [43] 744.30 233.97 1513/1129009 _sleep [150] 906.15 284.84 1842/1129009 dastrategy [116] 1170.32 367.88 2379/1129009 mps_intr_msi [194] 1329.22 417.83 2702/1129009 ixgbe_handle_que [56] 15078.43 4739.79 30651/1129009 nfsrc_unlock [174] 25869.62 8131.91 52587/1129009 _vm_map_lock [129] 44082.69 13857.05 89610/1129009 nfsrvd_sentcache [74] 113077.19 35544.94 229860/1129009 nfsrc_trimcache [26] 149533.34 47004.65 303967/1129009 nfsrvd_getcache [14] 198698.26 62459.27 403908/1129009 nfsrvd_updatecache [9] [5] 16.4 555404.00 174586.97 1129009 _mtx_lock_sleep [5] 9074.71 139536.06 87523/87603 turnstile_trywait [45] 3085.07 11049.12 3870/3964 turnstile_wait [207] 166.20 10020.76 4390/4728 lockstat_nsecs [229] 505.86 276.39 322/324 turnstile_cancel [552] 516.23 123.24 384/341743 spinlock_exit [136] 161.51 0.00 61/9744 spinlock_enter [156] 44.18 0.00 16/6592 critical_enter [190] 23.20 0.00 4/35 turnstile_setowner [767] 1.35 3.10 1/1531 propagate_priority [280] ----------------------------------------------- 13199.00 504436.33 584319/584319 nfssvc_program [4] [9] 11.6 13199.00 504436.33 584319 nfsrvd_updatecache [9] 198698.26 62459.27 403908/1129009 _mtx_lock_sleep [5] 107453.19 94387.86 212423/400403 nfsrc_trimcache [26] 1974.56 16386.18 2411/3646 nfsrc_freecache [148] 120.15 10877.79 1393/3281 _mtx_unlock_sleep [154] 139.67 10006.86 2/12 lapic_handle_intr [93] 257.17 946.79 431/6821 m_copym [186] 539.54 0.00 124/26922 bcopy [53] 2.07 31.89 20/87603 turnstile_trywait [45] 10.68 23.27 3/24500 free [102] 25.54 6.10 19/341743 spinlock_exit [136] 28.33 0.00 10/36 turnstile_chain_unlock [844] 13.58 10.56 8/30514 uma_zalloc_arg [70] 18.35 0.00 7/37 nfsrc_wanted [851] 10.90 0.83 5/284 turnstile_lookup [582] 0.11 6.85 3/4728 lockstat_nsecs [229] ----------------------------------------------- 23075.00 403396.18 468009/468009 nfssvc_program [4] [14] 9.6 23075.00 403396.18 468009 nfsrvd_getcache [14] 149533.34 47004.65 303967/1129009 _mtx_lock_sleep [5] 95069.59 83510.00 187942/400403 nfsrc_trimcache [26] 1104.58 13395.38 993/8846 malloc [47] 117.22 10612.28 1359/3281 _mtx_unlock_sleep [154] 1570.18 1088.69 1016/2145 in_cksum_skip [298] 134.35 0.00 27/9682 bzero [105] 127.38 0.00 46/1196 in_cksumdata [347] 28.23 6.74 21/341743 spinlock_exit [136] 1.87 28.70 18/87603 turnstile_trywait [45] 25.50 0.00 9/36 turnstile_chain_unlock [844] 11.88 9.24 7/30514 uma_zalloc_arg [70] 13.08 0.99 6/284 turnstile_lookup [582] 0.04 2.28 1/4728 lockstat_nsecs [229] ----------------------------------------------- 19.22 16.88 38/400403 nfssvc_program [4] 95069.59 83510.00 187942/400403 nfsrvd_getcache [14] 107453.19 94387.86 212423/400403 nfsrvd_updatecache [9] [26] 8.5 202542.00 177914.74 400403 nfsrc_trimcache [26] 113077.19 35544.94 229860/1129009 _mtx_lock_sleep [5] 657.00 19128.82 5754/5754 nfsrv_checksockseqnum [183] 1011.44 8393.58 1235/3646 nfsrc_freecache [148] 14.25 31.02 4/24500 free [102] 2.48 28.45 5/16831 m_freem [59] 0.93 14.35 9/87603 turnstile_trywait [45] 7.86 0.00 3/37 nfsrc_wanted [851] 1.57 0.86 1/324 turnstile_cancel [552] ----------------------------------------------- The full gprof output for the kernel is at . An amazing visualization of this (thanks to gprof2dot.py and dot) is at . There are some functions missing from the visualization because the gprof buckets overflowed. Is there a documented way to merge the kernel and ZFS gprof data output by pmcstat(1) so that those functions will be properly accounted for? -GAWollman From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 2 05:17:02 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C499A106566B for ; Tue, 2 Oct 2012 05:17:02 +0000 (UTC) (envelope-from tim@kientzle.com) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id 972508FC12 for ; Tue, 2 Oct 2012 05:17:02 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id q925Gr9K084914; Tue, 2 Oct 2012 05:16:53 GMT (envelope-from tim@kientzle.com) Received: from [192.168.2.143] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id cq7qbx5d328pv8g7sahzfn5raa; Tue, 02 Oct 2012 05:16:53 +0000 (UTC) (envelope-from tim@kientzle.com) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=iso-8859-1 From: Tim Kientzle In-Reply-To: <5069C9FC.6020400@brandonfa.lk> Date: Mon, 1 Oct 2012 22:16:53 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <87549776-9051-4B4B-8D53-DAE6D51C2A94@kientzle.com> References: <5069C9FC.6020400@brandonfa.lk> To: Brandon Falk X-Mailer: Apple Mail (2.1278) Cc: freebsd-hackers@freebsd.org Subject: Re: SMP Version of tar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 05:17:02 -0000 On Oct 1, 2012, at 9:51 AM, Brandon Falk wrote: > I would be willing to work on a SMP version of tar (initially just = gzip or something). >=20 > I don't have the best experience in compression, and how to = multi-thread it, but I think I would be able to learn and help out. >=20 > Note: I would like to make this for *BSD under the BSD license. I am = aware that there are already tools to do this (under GPL), but I would = really like to see this existent in the FreeBSD base. >=20 > Anyone interested? Great! First rule: be skeptical. In particular, tar is so entirely disk-bound = that many performance optimizations have no impact whatsoever. If you = don't do a lot of testing, you can end up wasting a lot of time. There are a few different parallel command-line compressors and = decompressors in ports; experiment a lot (with large files being read = from and/or written to disk) and see what the real effect is. In = particular, some decompression algorithms are actually faster than = memcpy() when run on a single processor. Parallelizing such algorithms = is not likely to help much in the real world. The two popular algorithms I would expect to benefit most are bzip2 = compression and lzma compression (targeting xz or lzip format). For = decompression, bzip2 is block-oriented so fits SMP pretty naturally. = Other popular algorithms are stream-oriented and less amenable to = parallelization. Take a careful look at pbzip2, which is a parallelized bzip2/bunzip2 = implementation that's already under a BSD license. You should be able = to get a lot of ideas about how to implement a parallel compression = algorithm. Better yet, you might be able to reuse a lot of the existing = pbzip2 code. Mark Adler's pigz is also worth studying. It's also license-friendly, = and is built on top of regular zlib, which is a nice technique when it's = feasible. There are three fundamentally different implementation approaches with = different complexity/performance issues: * Implement as a stand-alone executable similar to pbzip2. This makes = your code a lot simpler and makes it reasonably easy for people to reuse = your work. This could work with tar, though it could be slightly slower = than the in-process version due to the additional data-copying and = process-switch overhead. * Implement within libarchive directly. This would benefit tar and a = handful of other programs that use libarchive, but may not be worth the = complexity. * Implement as a standalone library with an interface similar to zlib = or libbz2 or liblzma. The last would be my personal preference, though it's probably the most = complex of all. That would easily support libarchive and you could = create a simple stand-alone wrapper around it as well, giving you the = best of all worlds. If you could extend the pigz technique, you might be able to build a = multi-threaded compression library where the actual compression was = handled by an existing single-threaded library. Since zlib, bzlib, and = liblzma already have similar interfaces, your layer might require only a = thin adapter to handle any of those three. *That* would be very = interesting, indeed. Sounds like a fun project. I wish I had time to work on something like = this. Cheers, Tim From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 2 06:36:44 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 47C611065674 for ; Tue, 2 Oct 2012 06:36:44 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from mail.yamagi.org (mail.yamagi.org [IPv6:2a01:4f8:121:2102:1::7]) by mx1.freebsd.org (Postfix) with ESMTP id CFCF08FC0A for ; Tue, 2 Oct 2012 06:36:43 +0000 (UTC) Received: from happy.home.yamagi.org (hmbg-5f77e48d.pool.mediaWays.net [95.119.228.141]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.yamagi.org (Postfix) with ESMTPSA id 29B061666312; Tue, 2 Oct 2012 08:36:40 +0200 (CEST) Date: Tue, 2 Oct 2012 08:36:34 +0200 From: Yamagi Burmeister To: tim@kientzle.com Message-Id: <20121002083634.3103fe958508a4026384ac96@yamagi.org> In-Reply-To: <87549776-9051-4B4B-8D53-DAE6D51C2A94@kientzle.com> References: <5069C9FC.6020400@brandonfa.lk> <87549776-9051-4B4B-8D53-DAE6D51C2A94@kientzle.com> X-Mailer: Sylpheed 3.2.0 (GTK+ 2.24.6; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="PGP-SHA1"; boundary="Signature=_Tue__2_Oct_2012_08_36_34_+0200_EDLuBl3TxLr/FPZX" Cc: freebsd-hackers@freebsd.org, bfalk_bsd@brandonfa.lk Subject: Re: SMP Version of tar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 06:36:44 -0000 --Signature=_Tue__2_Oct_2012_08_36_34_+0200_EDLuBl3TxLr/FPZX Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, 1 Oct 2012 22:16:53 -0700 Tim Kientzle wrote: > There are a few different parallel command-line compressors and decompres= sors in ports; experiment a lot (with large files being read from and/or wr= itten to disk) and see what the real effect is. In particular, some decomp= ression algorithms are actually faster than memcpy() when run on a single p= rocessor. Parallelizing such algorithms is not likely to help much in the = real world. >=20 > The two popular algorithms I would expect to benefit most are bzip2 compr= ession and lzma compression (targeting xz or lzip format). For decompressi= on, bzip2 is block-oriented so fits SMP pretty naturally. Other popular al= gorithms are stream-oriented and less amenable to parallelization. >=20 > Take a careful look at pbzip2, which is a parallelized bzip2/bunzip2 impl= ementation that's already under a BSD license. You should be able to get a= lot of ideas about how to implement a parallel compression algorithm. Bet= ter yet, you might be able to reuse a lot of the existing pbzip2 code. >=20 > Mark Adler's pigz is also worth studying. It's also license-friendly, an= d is built on top of regular zlib, which is a nice technique when it's feas= ible. Just a small note: There's a parallel implementation of xz called "pixz". It's build atop of liblzma and libarchiv and stands under a=20 BSD style license. See: https://github.com/vasi/pixz Maybe it's possible to reuse most of the code. --=20 Homepage: www.yamagi.org XMPP: yamagi@yamagi.org GnuPG/GPG: 0xEFBCCBCB --Signature=_Tue__2_Oct_2012_08_36_34_+0200_EDLuBl3TxLr/FPZX Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlBqi3cACgkQWTjlg++8y8u5uwCfWMSoOYdxB2ZDV+qDbfEX3z76 6kcAn0bJRHV47a4i2po55UTQKnrXBulb =fMmx -----END PGP SIGNATURE----- --Signature=_Tue__2_Oct_2012_08_36_34_+0200_EDLuBl3TxLr/FPZX-- From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 2 07:06:32 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9EB281065673 for ; Tue, 2 Oct 2012 07:06:32 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 70D158FC0A for ; Tue, 2 Oct 2012 07:06:32 +0000 (UTC) Received: by pbbrp8 with SMTP id rp8so9807714pbb.13 for ; Tue, 02 Oct 2012 00:06:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=46j0nzhhi2eAr2aH0kFi4GSsoJw1aS+r637ll3NYwvQ=; b=aK6tLc2vADqVTLbE0XlTl/GLKUyRBopGE4FuvrU2O+zYbBMwpGcG6tVb6aHiuoHdj7 VhLKaC6ITFxCauSe61WyYiVfiEKajoH13RNla+K2YEzmj/LsiT4CkZjv3qZplpzsiMcj UZ6cRME9Ea8Q5nbxihJUH/yxWNlS+JmGa3ddT+3oIzV0e+3injfW8eCW7+GQNaaGLVKg PZVAmEAGaky6eF4CJU72dmVOrhc1zjpuxd2EfqrUEswIK5IFMml/aiDMJ8fmvuBhcboE FINBw7pi/7vQZrveeKbH0/at3icrUgDzjQ79lgh3akwjq4Opz20C9/ITIpft1CY8uvaJ 3wDg== MIME-Version: 1.0 Received: by 10.66.78.73 with SMTP id z9mr42294926paw.9.1349161591639; Tue, 02 Oct 2012 00:06:31 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.68.16.40 with HTTP; Tue, 2 Oct 2012 00:06:31 -0700 (PDT) In-Reply-To: <20121002083634.3103fe958508a4026384ac96@yamagi.org> References: <5069C9FC.6020400@brandonfa.lk> <87549776-9051-4B4B-8D53-DAE6D51C2A94@kientzle.com> <20121002083634.3103fe958508a4026384ac96@yamagi.org> Date: Tue, 2 Oct 2012 00:06:31 -0700 X-Google-Sender-Auth: CJsyMpA5UncLIQBR8pUdQILT9Ys Message-ID: From: Adrian Chadd To: Yamagi Burmeister Content-Type: text/plain; charset=ISO-8859-1 Cc: bfalk_bsd@brandonfa.lk, freebsd-hackers@freebsd.org Subject: Re: SMP Version of tar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 07:06:32 -0000 .. please keep in mind that embedded platforms (a) don't necessarily benefit from it, and (b) have a very small footprint. Bloating out the compression/archival tools for the sake of possible SMP support will make me very, very sad. Adrian From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 2 07:36:21 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C8E9E106564A for ; Tue, 2 Oct 2012 07:36:21 +0000 (UTC) (envelope-from bfalk_bsd@brandonfa.lk) Received: from mail-qa0-f54.google.com (mail-qa0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 723498FC0A for ; Tue, 2 Oct 2012 07:36:21 +0000 (UTC) Received: by qady23 with SMTP id y23so316066qad.13 for ; Tue, 02 Oct 2012 00:36:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=brandonfa.lk; s=google; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=bZMwA3sFPyUVU9N5//fsrmpK7BsbNXlUb3D6OKtDTis=; b=W8cPEyZfi1iVCrgs1NxPIgw/1CBg1BLmZUZTAPMITEU7AOEfyrJI3CZ/fbCbXZGTki hU53HtRLRUuYokFxxDR7AaK1+gyQbk84lR52gEr97f5Lz0VjVct9qgLOrbYxYzlcu12A FLcYThYqhdS3hRQXnoE0VObIaB71gzKDAWxKY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding :x-gm-message-state; bh=bZMwA3sFPyUVU9N5//fsrmpK7BsbNXlUb3D6OKtDTis=; b=aS6VWIVDuyWcwwvTcnoPVhKd1grXNmHM+fEUfaG7M1fhCSo6Fk+uWhShhQZMaBSxj1 rcEQ0h74BcmQnNQMSRtaBfNJ9fCNrxepF6Jr2r5KElLZPJ28AUmddhjybuUd3b8B6/YZ M/lWTEKzh3C1KRYcIgITRdllZpCUuUQTV060C5/a/iudZlL5yJ/HbZqyA62qefnvWZUy NbmeLWoA5Jl3N3YugW77ElIZBK8E2A1Zyar5hYSu6xNKRweHCsAtRNQzGrjWNBF2g4yC BjwqCjgtj2YeECi7OT6GrmRBN8DFv4xKnCdOEoE/vdjFwaXMcVxWummbmuDJfpgz4yB7 YrBg== Received: by 10.224.196.132 with SMTP id eg4mr973236qab.93.1349163380501; Tue, 02 Oct 2012 00:36:20 -0700 (PDT) Received: from [10.64.64.96] (pool-72-83-231-212.washdc.fios.verizon.net. [72.83.231.212]) by mx.google.com with ESMTPS id b5sm774034qao.13.2012.10.02.00.36.19 (version=SSLv3 cipher=OTHER); Tue, 02 Oct 2012 00:36:19 -0700 (PDT) Message-ID: <506A9973.6000507@brandonfa.lk> Date: Tue, 02 Oct 2012 03:36:19 -0400 From: Brandon Falk User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 To: Adrian Chadd References: <5069C9FC.6020400@brandonfa.lk> <87549776-9051-4B4B-8D53-DAE6D51C2A94@kientzle.com> <20121002083634.3103fe958508a4026384ac96@yamagi.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQmohTrl8S/dPHRkhfjAKdfKr3Aft3t5VXS/wMATQmS0+jsVbBY7FTJPRL3ITOHMghSvOdRa Cc: Yamagi Burmeister , freebsd-hackers@freebsd.org Subject: Re: SMP Version of tar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 07:36:21 -0000 Don't worry. I'm well known to over-optimize for both size and speed. I have an old Pentium 3 800MHz single core that I can use to simulate an embedded device (well, a decently powered one), to verify that I'm not killing the single-core performance (I could add CPU capability detection to help with that). Also, I still need to find time to do all this research and development. -Brandon On 10/2/2012 3:06 AM, Adrian Chadd wrote: > .. please keep in mind that embedded platforms (a) don't necessarily > benefit from it, and (b) have a very small footprint. Bloating out the > compression/archival tools for the sake of possible SMP support will > make me very, very sad. > > > > Adrian From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 2 10:59:26 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E435106564A for ; Tue, 2 Oct 2012 10:59:26 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from mo6-p00-ob.rzone.de (mo6-p00-ob.rzone.de [IPv6:2a01:238:20a:202:5300::1]) by mx1.freebsd.org (Postfix) with ESMTP id E23CB8FC14 for ; Tue, 2 Oct 2012 10:59:25 +0000 (UTC) X-RZG-AUTH: :JiIXek6mfvEEUpFQdo7Fj1/zg48CFjWjQv0cW+St/nW/YfgwsCVviY+VFsLulA== X-RZG-CLASS-ID: mo00 Received: from britannica.bec.de (ip-90-186-5-200.web.vodafone.de [90.186.5.200]) by smtp.strato.de (josoe mo50) (RZmta 30.20 DYNA|AUTH) with (AES128-SHA encrypted) ESMTPA id z00223o929k1LX for ; Tue, 2 Oct 2012 12:59:23 +0200 (CEST) Received: by britannica.bec.de (sSMTP sendmail emulation); Tue, 02 Oct 2012 12:59:21 +0200 Date: Tue, 2 Oct 2012 12:59:21 +0200 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org Message-ID: <20121002105921.GA10034@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org References: <5069C9FC.6020400@brandonfa.lk> <87549776-9051-4B4B-8D53-DAE6D51C2A94@kientzle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87549776-9051-4B4B-8D53-DAE6D51C2A94@kientzle.com> User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: SMP Version of tar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 10:59:26 -0000 On Mon, Oct 01, 2012 at 10:16:53PM -0700, Tim Kientzle wrote: > * Implement within libarchive directly. This would benefit tar and > a handful of other programs that use libarchive, but may not be > worth the complexity. The complexity shouldn't actually be that bad. Basically, use a large internal buffer in the compression filter and make the queueing for distribution to threads an implementation detail. Alternatively, just provide a compatible implementation of libz's event interface. Joerg From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 2 12:28:36 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B39DA106564A; Tue, 2 Oct 2012 12:28:36 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id E44BE8FC15; Tue, 2 Oct 2012 12:28:35 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap4EAHLdalCDaFvO/2dsb2JhbABFFoV1uVaCIAEBBAEjVhsYAgINGQJZBogSBgumToI7kG6BIYoZhRmBEgOVaYEVjxaDA4E/CDQ X-IronPort-AV: E=Sophos;i="4.80,523,1344225600"; d="scan'208";a="184184041" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 02 Oct 2012 08:28:29 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 1D258B3F85; Tue, 2 Oct 2012 08:28:29 -0400 (EDT) Date: Tue, 2 Oct 2012 08:28:29 -0400 (EDT) From: Rick Macklem To: Garrett Wollman Message-ID: <499414315.1544891.1349180909058.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20586.27582.478147.838896@hergotha.csail.mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: rmacklem@freebsd.org, hackers@freebsd.org, jkoshy@freebsd.org Subject: Re: NFS server bottlenecks X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 12:28:36 -0000 Garrett Wollman wrote: > I had an email conversation with Rick Macklem about six months ago > about NFS server bottlenecks. I'm now in a position to observe my > large-scale NFS server under an actual production load, so I thought I > would update folks on what it looks like. This is a 9.1 prerelease > kernel (I hope 9.1 will be released soon as I have four moe of these > servers to deploy!). When under nearly 100% load on an 8-core > (16-thread) Quanta QSSC-S99Q storage server, with a 10G network > interface, pmcstat tells me this: > > PMC: [INST_RETIRED.ANY_P] Samples: 2727105 (100.0%) , 27 unresolved > Key: q => exiting... > %SAMP IMAGE FUNCTION CALLERS > 29.3 kernel _mtx_lock_sleep nfsrvd_updatecache:10.0 > nfsrvd_getcache:7.4 ... > 9.5 kernel cpu_search_highest cpu_search_highest:8.1 sched_idletd:1.4 > 7.4 zfs.ko lzjb_decompress zio_decompress > 4.3 kernel _mtx_lock_spin turnstile_trywait:2.2 pmclog_reserve:1.0 ... > 4.0 zfs.ko fletcher_4_native zio_checksum_error:3.1 > zio_checksum_compute:0.8 > 3.6 kernel cpu_search_lowest cpu_search_lowest > 3.3 kernel nfsrc_trimcache nfsrvd_getcache:1.6 nfsrvd_updatecache:1.6 > 2.3 kernel ipfw_chk ipfw_check_hook > 2.1 pmcstat _init > 1.1 kernel _sx_xunlock > 0.9 kernel _sx_xlock > 0.9 kernel spinlock_exit > > This does seem to confirm my original impression that the NFS replay > cache is quite expensive. Running a gprof(1) analysis on the same PMC > data reveals a bit more detail (I've removed some uninteresting parts > of the call graph): > I can't remember (I am early retired now;-) if I mentioned this patch before: http://people.freebsd.org/~rmacklem/drc.patch It adds tunables vfs.nfsd.tcphighwater and vfs.nfsd.udphighwater that can be twiddled so that the drc is trimmed less frequently. By making these values larger, the trim will only happen once/sec until the high water mark is reached, instead of on every RPC. The tradeoff is that the DRC will become larger, but given memory sizes these days, that may be fine for you. jwd@ was going to test it, but he moved to a different job away from NFS, so the patch has just been collecting dust. If you could test it, that would be nice, rick ps: Also, the current patch still locks before checking if it needs to do the trim. I think that could safely be changed so that it doesn't lock/unlock when it isn't doing the trim, if that makes a significant difference. > > called/total parents > index %time self descendents called+self name index > called/total children > 4881.00 2004642.70 932627/932627 svc_run_internal [2] > [4] 45.1 4881.00 2004642.70 932627 nfssvc_program [4] > 13199.00 504436.33 584319/584319 nfsrvd_updatecache [9] > 23075.00 403396.18 468009/468009 nfsrvd_getcache [14] > 1032.25 416249.44 2239/2284 svc_sendreply_mbuf [15] > 6168.00 381770.44 11618/11618 nfsrvd_dorpc [24] > 3526.87 86869.88 112478/112514 nfsrvd_sentcache [74] > 890.00 50540.89 4252/4252 svc_getcred [101] > 14876.60 32394.26 4177/24500 crfree [263] > 11550.11 25150.73 3243/24500 free [102] > 1348.88 15451.66 2716/16831 m_freem [59] > 4066.61 216.81 1434/1456 svc_freereq [321] > 2342.15 677.40 557/1459 malloc_type_freed [265] > 59.14 1916.84 134/2941 crget [113] > 1602.25 0.00 322/9682 bzero [105] > 690.93 0.00 43/44 getmicrotime [571] > 287.22 7.33 138/1205 prison_free [384] > 233.61 0.00 60/798 PHYS_TO_VM_PAGE [358] > 203.12 0.00 94/230 nfsrv_mallocmget_limit [632] > 151.76 0.00 51/1723 pmap_kextract [309] > 0.78 70.28 9/3281 _mtx_unlock_sleep [154] > 19.22 16.88 38/400403 nfsrc_trimcache [26] > 11.05 21.74 7/197 crsetgroups [532] > 30.37 0.00 11/6592 critical_enter [190] > 25.50 0.00 9/36 turnstile_chain_unlock [844] > 24.86 0.00 3/7 nfsd_errmap [913] > 12.36 8.57 8/2145 in_cksum_skip [298] > 9.10 3.59 5/12455 mb_free_ext [140] > 1.84 4.85 2/2202 VOP_UNLOCK_APV [269] > > ----------------------------------------------- > > 0.49 0.15 1/1129009 uhub_explore [1581] > 0.49 0.15 1/1129009 tcp_output [10] > 0.49 0.15 1/1129009 pmap_remove_all [1141] > 0.49 0.15 1/1129009 vm_map_insert [236] > 0.49 0.15 1/1129009 vnode_create_vobject [281] > 0.49 0.15 1/1129009 biodone [351] > 0.49 0.15 1/1129009 vm_object_madvise [670] > 0.49 0.15 1/1129009 xpt_done [483] > 0.49 0.15 1/1129009 vputx [80] > 0.49 0.15 1/1129009 vm_map_delete [49] > 0.49 0.15 1/1129009 vm_object_deallocate [356] > 0.49 0.15 1/1129009 vm_page_unwire [338] > 0.49 0.15 1/1129009 pmap_change_wiring [318] > 0.98 0.31 2/1129009 getnewvnode [227] > 0.98 0.31 2/1129009 pmap_clear_reference [1004] > 0.98 0.31 2/1129009 usbd_do_request_flags [1282] > 0.98 0.31 2/1129009 vm_object_collapse [587] > 0.98 0.31 2/1129009 vm_object_page_remove [122] > 1.48 0.46 3/1129009 mpt_pci_intr [487] > 1.48 0.46 3/1129009 pmap_extract [355] > 1.48 0.46 3/1129009 vm_fault_unwire [171] > 1.97 0.62 4/1129009 vgonel [270] > 1.97 0.62 4/1129009 vm_object_shadow [926] > 1.97 0.62 4/1129009 zone_alloc_item [434] > 2.46 0.77 5/1129009 vnlru_free [235] > 2.46 0.77 5/1129009 insmntque1 [737] > 2.95 0.93 6/1129009 zone_free_item [409] > 3.94 1.24 8/1129009 pmap_enter_object [805] > 4.92 1.55 10/1129009 vfs_ref [722] > 4.92 1.55 10/1129009 pmap_clear_modify [814] > 5.41 1.70 11/1129009 vn_isdisk [391] > 5.90 1.86 12/1129009 rtalloc1_fib [158] > 5.90 1.86 12/1129009 soreceive_generic [99] > 6.40 2.01 13/1129009 pmap_remove_pages [403] > 6.89 2.16 14/1129009 _mtx_lock_flags [565] > 6.89 2.16 14/1129009 random_yarrow_read [542] > 6.89 2.16 14/1129009 nfsrv_checkgetattr [1133] > 7.38 2.32 15/1129009 svc_vc_recv [55] > 7.38 2.32 15/1129009 ixgbe_rxeof [12] > 8.85 2.78 18/1129009 _cv_timedwait_sig [859] > 9.35 2.94 19/1129009 vref [242] > 10.82 3.40 22/1129009 pmap_enter_quick [690] > 11.31 3.56 23/1129009 tcp_do_segment [35] > 13.28 4.18 27/1129009 sosend_generic [25] > 15.74 4.95 32/1129009 g_io_schedule_down [76] > 19.19 6.03 39/1129009 vn_start_write [400] > 20.66 6.49 42/1129009 ip_output [32] > 23.12 7.27 47/1129009 pmap_copy [467] > 23.12 7.27 47/1129009 __mnt_vnode_next_all [520] > 29.52 9.28 60/1129009 vm_page_free_toq [149] > 30.50 9.59 62/1129009 vm_fault_hold [268] > 36.90 11.60 75/1129009 g_io_request [172] > 38.37 12.06 78/1129009 prison_hold [330] > 38.86 12.22 79/1129009 taskqueue_member [423] > 39.36 12.37 80/1129009 vm_page_deactivate [812] > 39.85 12.53 81/1129009 vm_page_activate [790] > 40.83 12.83 83/1129009 in_matroute [197] > 45.75 14.38 93/1129009 vm_page_alloc [128] > 48.70 15.31 99/1129009 prison_free [384] > 66.90 21.03 136/1129009 vholdl [275] > 73.30 23.04 149/1129009 vfs_unbusy [470] > 82.65 25.98 168/1129009 key_addref [233] > 85.60 26.91 174/1129009 vn_finished_write [407] > 91.50 28.76 186/1129009 _key_freesp [286] > 92.48 29.07 188/1129009 vfs_busy [288] > 102.32 32.16 208/1129009 _vm_object_allocate [738] > 107.24 33.71 218/1129009 taskqueue_enqueue [68] > 112.16 35.26 228/1129009 xprt_active [267] > 125.94 39.59 256/1129009 vm_pageq_remove [650] > 128.89 40.51 262/1129009 svc_freereq [321] > 128.89 40.51 262/1129009 pmap_remove [106] > 153.49 48.25 312/1129009 vdropl [251] > 154.96 48.71 315/1129009 vfs_busyfs [239] > 180.05 56.60 366/1129009 pmap_enter [245] > 182.02 57.22 370/1129009 svc_run_internal [2] > 213.50 67.11 434/1129009 ixgbe_msix_que [11] > 248.43 78.09 505/1129009 key_allocsp [221] > 255.32 80.26 519/1129009 camisr [107] > 373.87 117.52 760/1129009 ipmi_poll [635] > 404.87 127.27 823/1129009 g_io_deliver [162] > 440.78 138.56 896/1129009 g_io_schedule_up [137] > 462.91 145.51 941/1129009 taskqueue_run_locked [43] > 744.30 233.97 1513/1129009 _sleep [150] > 906.15 284.84 1842/1129009 dastrategy [116] > 1170.32 367.88 2379/1129009 mps_intr_msi [194] > 1329.22 417.83 2702/1129009 ixgbe_handle_que [56] > 15078.43 4739.79 30651/1129009 nfsrc_unlock [174] > 25869.62 8131.91 52587/1129009 _vm_map_lock [129] > 44082.69 13857.05 89610/1129009 nfsrvd_sentcache [74] > 113077.19 35544.94 229860/1129009 nfsrc_trimcache [26] > 149533.34 47004.65 303967/1129009 nfsrvd_getcache [14] > 198698.26 62459.27 403908/1129009 nfsrvd_updatecache [9] > [5] 16.4 555404.00 174586.97 1129009 _mtx_lock_sleep [5] > 9074.71 139536.06 87523/87603 turnstile_trywait [45] > 3085.07 11049.12 3870/3964 turnstile_wait [207] > 166.20 10020.76 4390/4728 lockstat_nsecs [229] > 505.86 276.39 322/324 turnstile_cancel [552] > 516.23 123.24 384/341743 spinlock_exit [136] > 161.51 0.00 61/9744 spinlock_enter [156] > 44.18 0.00 16/6592 critical_enter [190] > 23.20 0.00 4/35 turnstile_setowner [767] > 1.35 3.10 1/1531 propagate_priority [280] > > ----------------------------------------------- > > 13199.00 504436.33 584319/584319 nfssvc_program [4] > [9] 11.6 13199.00 504436.33 584319 nfsrvd_updatecache [9] > 198698.26 62459.27 403908/1129009 _mtx_lock_sleep [5] > 107453.19 94387.86 212423/400403 nfsrc_trimcache [26] > 1974.56 16386.18 2411/3646 nfsrc_freecache [148] > 120.15 10877.79 1393/3281 _mtx_unlock_sleep [154] > 139.67 10006.86 2/12 lapic_handle_intr [93] > 257.17 946.79 431/6821 m_copym [186] > 539.54 0.00 124/26922 bcopy [53] > 2.07 31.89 20/87603 turnstile_trywait [45] > 10.68 23.27 3/24500 free [102] > 25.54 6.10 19/341743 spinlock_exit [136] > 28.33 0.00 10/36 turnstile_chain_unlock [844] > 13.58 10.56 8/30514 uma_zalloc_arg [70] > 18.35 0.00 7/37 nfsrc_wanted [851] > 10.90 0.83 5/284 turnstile_lookup [582] > 0.11 6.85 3/4728 lockstat_nsecs [229] > > ----------------------------------------------- > > 23075.00 403396.18 468009/468009 nfssvc_program [4] > [14] 9.6 23075.00 403396.18 468009 nfsrvd_getcache [14] > 149533.34 47004.65 303967/1129009 _mtx_lock_sleep [5] > 95069.59 83510.00 187942/400403 nfsrc_trimcache [26] > 1104.58 13395.38 993/8846 malloc [47] > 117.22 10612.28 1359/3281 _mtx_unlock_sleep [154] > 1570.18 1088.69 1016/2145 in_cksum_skip [298] > 134.35 0.00 27/9682 bzero [105] > 127.38 0.00 46/1196 in_cksumdata [347] > 28.23 6.74 21/341743 spinlock_exit [136] > 1.87 28.70 18/87603 turnstile_trywait [45] > 25.50 0.00 9/36 turnstile_chain_unlock [844] > 11.88 9.24 7/30514 uma_zalloc_arg [70] > 13.08 0.99 6/284 turnstile_lookup [582] > 0.04 2.28 1/4728 lockstat_nsecs [229] > > ----------------------------------------------- > > 19.22 16.88 38/400403 nfssvc_program [4] > 95069.59 83510.00 187942/400403 nfsrvd_getcache [14] > 107453.19 94387.86 212423/400403 nfsrvd_updatecache [9] > [26] 8.5 202542.00 177914.74 400403 nfsrc_trimcache [26] > 113077.19 35544.94 229860/1129009 _mtx_lock_sleep [5] > 657.00 19128.82 5754/5754 nfsrv_checksockseqnum [183] > 1011.44 8393.58 1235/3646 nfsrc_freecache [148] > 14.25 31.02 4/24500 free [102] > 2.48 28.45 5/16831 m_freem [59] > 0.93 14.35 9/87603 turnstile_trywait [45] > 7.86 0.00 3/37 nfsrc_wanted [851] > 1.57 0.86 1/324 turnstile_cancel [552] > > ----------------------------------------------- > > The full gprof output for the kernel is at > . > An amazing visualization of this (thanks to gprof2dot.py and dot) is > at > . > There are some functions missing from the visualization because the > gprof buckets overflowed. > > Is there a documented way to merge the kernel and ZFS gprof data > output by pmcstat(1) so that those functions will be properly > accounted for? > > -GAWollman From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 2 12:40:53 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E4F91065670; Tue, 2 Oct 2012 12:40:53 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 00C1E8FC0C; Tue, 2 Oct 2012 12:40:53 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 67421B911; Tue, 2 Oct 2012 08:40:52 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Tue, 2 Oct 2012 07:50:23 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p20; KDE/4.5.5; amd64; ; ) References: <20121001223100.E7D0D58093@chaos.jnpr.net> In-Reply-To: <20121001223100.E7D0D58093@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201210020750.23358.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 02 Oct 2012 08:40:52 -0400 (EDT) Cc: Garrett Cooper , freebsd-hackers@freebsd.org, "Simon J. Gerraty" Subject: Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 12:40:53 -0000 On Monday, October 01, 2012 6:31:00 pm Simon J. Gerraty wrote: > Hi Garrett, > > >> From: Garrett Cooper > >> Subject: [CFT/RFC]: refactor bsd.prog.mk to understand multiple = > >programs instead of a singular program > >> Date: September 2, 2012 11:01:09 PM PDT > >> To: freebsd-hackers@freebsd.org > >> Cc: "freebsd-arch@FreeBSD.org Arch" > >>=20 > >> Hello, > >> I've been a bit busy working on porting over ATF from NetBSD, and > > Thanks, we've been using ATF in Junos for a while and glad to see it > being imported to FreeBSD. > > >> one of the pieces that's currently not available in FreeBSD that's > >> available in NetBSD is the ability to understand and compile multiple > >> programs. In order to do this I had to refactor bsd.prog.mk (a lot). > > A change like this to bsd.prog.mk can have considerable fallout. > Eg. any makefile that tweaks OBJS is suddenly out of luck. > > Not to mention the fact that bsd.prog.mk goes from being relatively > simple, to unspeakably hard to read, and all for rather limited return. > > Apart from ATF, is there any huge demand for building multiple progs in > the same directory? > > FWIW we use progs.mk (as bsd.progs.mk) from > ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz > It isn't ideal, but it certainly avoids a lot of churn and complexity > for what is essentially a corner case. This sounds like a superior approach. It doesn't break any current use cases while giving the ability to build multiple programs in the few places that need it. It sounds like there are a few places under gnu/ from Garrett's reply that might be able to make use of this as well. BTW, one general comment. There seem to be two completely independent groups of folks working on ATF (e.g. there have been two different imports of ATF into the tree in two different locations IIRC, and now we have two different sets of patches to our system makefiles). Are these two groups talking to each other at all? I know in May that many folks (certainly multiple vendors) are interested in ATF, and it seems that both Juniper and Isilon have ported ATF internally. It seems that it might be good for the two groups to work together to avoid stomping on each other's toes. It seems there are some differences in the two approaches that merit working out to avoid a lot of wasted effort on both sides. Do we already have a freebsd-atf@ mailing list? If not, perhaps we should create one and start these discussions there? -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 2 14:19:56 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C5139106566C; Tue, 2 Oct 2012 14:19:56 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6E7048FC12; Tue, 2 Oct 2012 14:19:56 +0000 (UTC) Received: by oagn9 with SMTP id n9so6298531oag.13 for ; Tue, 02 Oct 2012 07:19:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=8qvwGc8uFPh6HD2amh6kNVgUUS/wI4OOAWBrKkyGk98=; b=CmMMZxxkO/bTiMulW8aiK8tWOwyKE//g0TKx9TaWN+CFs62js3fF5kCcsBUctod2KU F7Kb22ti53Fx5jx3FsYhjchShaO8JRo4I5m3PNJEaAjtcQsgUWS+z6nEtzF7+mE5j6+p yPGJpfOLuGBjcZtstm532ODKnDDe1GOPtkn/4NRS2xm8x2gSqeS3RrxWYAUGjuCL+vjH n40YjTVdXDaDsiSPWGZR/OpcVyfbTZYb0L5uJntXMi4fOeUEZjknWdd49oNID4dodW1V dP64oScFp6KWIBh49tk9vtZpgcWPAMubhOYU3J/o62tdDegWZtVzq+xhUv6nCkdk+OSd wzmw== MIME-Version: 1.0 Received: by 10.60.170.241 with SMTP id ap17mr14742956oec.4.1349187595757; Tue, 02 Oct 2012 07:19:55 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Tue, 2 Oct 2012 07:19:55 -0700 (PDT) In-Reply-To: <20121002000030.54CEE58093@chaos.jnpr.net> References: <20121001223100.E7D0D58093@chaos.jnpr.net> <20121002000030.54CEE58093@chaos.jnpr.net> Date: Tue, 2 Oct 2012 07:19:55 -0700 Message-ID: From: Garrett Cooper To: "Simon J. Gerraty" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 14:19:56 -0000 On Mon, Oct 1, 2012 at 5:00 PM, Simon J. Gerraty wrote: >>> Not to mention the fact that bsd.prog.mk goes from being relatively >>> simple, to unspeakably hard to read, and all for rather limited = >>return. > > This btw I think is the more important issue. > I was looking at bsd.prog.mk in netbsd the other day. > It has no business being that complex. Ok. Even though I originally thought that my changes were a bit hacky for not modifying bsd.man.mk, bsd.nls.mk, etc, looking back the way they were resolved was a little elegant in its simplicity (albeit not optimized). ... >>It order to do this, I need to be able to build multiple programs so = >>things are as transparent. On the flipside, bsd.prog[s].mk can live on = > > We put the test cases in a subdir of the lib/prog > This has multiple benefits, and eliminates any impact on the normal > build of said libs/progs. Hmmm... that's one of the 3 approaches I provided, but it turned out to be annoying to make this work at first inspection because of how things were done in our build system. Just for review (even though it's OT), the three approaches I presented were as follows... Note: in all 3 examples, clock.c is the source code and t_clock.c is the relevant unittest code. 1. Test programs live with the sources (this was the requested approach), e.g. lib/libc/gen/... .../clock.c .../t_clock.c 2. Test programs live in subdirs: lib/libc/gen/... .../clock.c .../tests/... .../t_clock.c 3. Test programs completely decoupled from the source tree: lib/libc/gen/... .../clock.c tests/lib/libc/gen/... .../t_clock.c A hybrid approach (1.+3. / 2.+3.) should probably be used anyhow for stress tests and the like that really have no business living in one particular directory in the source tree (sort of achieving what tools/regression does today, but hopefully in a less messy manner as tools/regression appears to have grown organically minus any single sane order). > Also a number of our teams find it necessary to create mock classes etc to > adequately test their libs. Keeping all this in a tests/ subdir makes > it easy, to keep things neat & up to date, and ensure that the tests > pass. Trying to do all that in the same dir as the lib would be ugly. This (consolidating everything under a single directory) is the way that was requested by the beforementioned parties. >>> FWIW we use progs.mk (as bsd.progs.mk) from >>> ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz=20 >>> It isn't ideal, but it certainly avoids a lot of churn and complexity >>> for what is essentially a corner case. >> >>This requires pulling bmake into FreeBSD proper in order for things to = >>function the last I checked as bsd.prog.mk depends upon bmake directives = > > This is already happening ;-) I wish I had known that. I pinged marcel/obrien about this some weeks ago and didn't receive a response. Originally bsd.progs.mk was going to be a home grown effort that was going to be killed off and replaced with NetBSD's build infrastructure, but I didn't get a reply and had to put this (my work to replace bsd.prog.mk) plan in motion given requests I was provided above for resolving the source/unittest code consolidation effort. >>Ideally however, I would like doing this compared to running a custom = >>build infrastructure, but (as you probably know) this requires = >>rototilling the current FreeBSD build system to a large degree = > > Actually building FreeBSD-current using bmake requires only modest changes. Yes for the most part, and I agree that bmake is definitely more advanced than FreeBSD pmake so I consider it a very welcome change. What are the plans for importing bmake into FreeBSD (this should probably be a different thread)? ... > I know, but it is a very useful thing to be able to do. > You can add knobs to control such things of course. I agree that it's a good thing (in theory) -- it'd just help to discuss this with Julio first because I know of a couple cases where this would be unusable given how bsd.test.mk is coded: 1. The "it works for me on my machine!" certification: It encourages environment tainting, which could be a really, really bad thing; I've seen developers take interesting shortcuts when testing their code (me included sometimes :)..); I've seen hardcoded paths, harcoded "paths" for named semaphores, things that "just work" because of someone's host environment, feature specific assumptions (developer X was doing testing on feature Y and things broke when he/she committed the testcase to head), etc where the "it works for me on my machine!" certification would be true more often than not. These same individuals would more likely than not execute things taking shortcuts, but I want to avoid creating a system where developers cut corners and commit too early because working within the confines of the system is not conducive to getting work done. 2. Failure in a subdirectory results in lost coverage: a/... .../b/... <- tests live here .../c/d/... <- tests live here If I execute make test from a/ (one of my goals), then it should iterate down a/b, then a/c/d and run the tests in a DFS style (BFS if invoked with -j > 1 -- AIIIEEEE). If a/b fails to execute the make process will bomb out because of how it executes shell statements. However, if I let a/b pass, then developers/testers will accidentally overlook failures, unless output is captured from the run and errors are searched for in the end. This isn't a super big problem if the tests are deterministic enough that fixing one test permits the others to run, but the lost test coverage in that period of time (or others if it transiently fails) could disguise other bugs. This would be fixed by changing bsd.test.mk to use atf-run properly, but that's work (not hard) that still needs to be done. (there are some other cases, but they're eluding me now) > Yes, but making one test.mk handle potentially lots of frameworks > will get rather ugly quite quickly. Understood and I agree. I wasn't advocating having a single, monolithic bsd.test.mk with all the test frameworks integrated into it (that would be silly and unmaintainable). > Better to add per-framework .mk files, and perhaps have them include a > bsd.test.mk which only handles high level logic rather than the details > of the frameworks. > > That's laregly why we didn't use bsd.test.mk Hmmmm... My goal was to make ATF a "one ring to rule them all" infrastructure so that way everything could be unified under ATF in one way, shape or form (even if it's just reporting), because lord knows we don't need a lua, unittest/nose, etc wrapper for reporting results. All of the jobs I've worked at write custom wrappers around testing infrastructures in order to present the data in a pretty format (HTML), but that's a largely wasted effort (unless someone REALLY wants to learn CSS/HTML/JS, which is an ongoing pain thanks to browser incompatibilities), but I realize that this is also more effort than I could get to right away, and it would have to be done as needed instead. Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 2 14:29:50 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A32AE106566B; Tue, 2 Oct 2012 14:29:50 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3D2808FC08; Tue, 2 Oct 2012 14:29:49 +0000 (UTC) Received: by obbwc20 with SMTP id wc20so6762440obb.13 for ; Tue, 02 Oct 2012 07:29:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=yMr25PEYOSjTSQbPntob1x4MGkqjurBBoNIbBfEp8ts=; b=W9/WfWyAB9+Z0jZc73axkCamQGvmnokjjnKbbt9o0aESQR1N/GXYSAKAsO2+piib6X 8aukgtyYcbo3tgGX4+BlEiVaN808Gdj64LuSX9zzXVK8tIhYjDxfeqwiRyv9eGnWVTPT sicfD+HEFaAz2ZthZ88tZ/zbnMmVwQG2KIPZIE58Ej8yAQlXr7ZAr+g5i2d0ho0uXWOa 3lw6Ltm98Fj5WM2HN8PJjge2hwXMojOTe4bXZ8D4JSP6njw1h2SYi8EEs7MnTaJQsZCz AUbKJStFLAOKEBkkjkqjJuN8XGzLDjU/oQe9/UO4Xb2VpGX7E+8zTSxAzwH9DBzvq3Hz wilw== MIME-Version: 1.0 Received: by 10.182.231.66 with SMTP id te2mr5485610obc.67.1349188189334; Tue, 02 Oct 2012 07:29:49 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Tue, 2 Oct 2012 07:29:49 -0700 (PDT) In-Reply-To: <201210020750.23358.jhb@freebsd.org> References: <20121001223100.E7D0D58093@chaos.jnpr.net> <201210020750.23358.jhb@freebsd.org> Date: Tue, 2 Oct 2012 07:29:49 -0700 Message-ID: From: Garrett Cooper To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, "Simon J. Gerraty" , freebsd-arch@freebsd.org Subject: Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 14:29:50 -0000 On Tue, Oct 2, 2012 at 4:50 AM, John Baldwin wrote: ... > This sounds like a superior approach. It doesn't break any current use > cases while giving the ability to build multiple programs in the few > places that need it. It sounds like there are a few places under gnu/ > from Garrett's reply that might be able to make use of this as well. For the record, gnu/cc/cc_tools/Makefile is where I first spotted a potential "bsd.progs.mk" candidate. Most of the other code doesn't care given how things are organized in our source tree. > BTW, one general comment. There seem to be two completely independent > groups of folks working on ATF (e.g. there have been two different > imports of ATF into the tree in two different locations IIRC, and now > we have two different sets of patches to our system makefiles). > > Are these two groups talking to each other at all? I know in May that > many folks (certainly multiple vendors) are interested in ATF, and it > seems that both Juniper and Isilon have ported ATF internally. It seems > that it might be good for the two groups to work together to avoid > stomping on each other's toes. It seems there are some differences in > the two approaches that merit working out to avoid a lot of wasted > effort on both sides. Both parties (Isilon/Juniper) are converging on the ATF porting work that Giorgos/myself have done after talking at the FreeBSD Foundation meet-n-greet. I have contributed all of the patches that I have other to marcel for feedback. > Do we already have a freebsd-atf@ mailing list? If not, perhaps we > should create one and start these discussions there? Probably wouldn't be a bad idea as I'm currently suspended a bit waiting on feedback for how to proceed; too bad freebsd-test is being used for other things :).. Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 2 15:39:21 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F192D106564A; Tue, 2 Oct 2012 15:39:20 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from exprod7og118.obsmtp.com (exprod7og118.obsmtp.com [64.18.2.8]) by mx1.freebsd.org (Postfix) with ESMTP id A385B8FC15; Tue, 2 Oct 2012 15:39:19 +0000 (UTC) Received: from P-EMHUB03-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob118.postini.com ([64.18.6.12]) with SMTP ID DSNKUGsKob/1aHEXgvXZAzyGrIgFecKw5o35@postini.com; Tue, 02 Oct 2012 08:39:20 PDT Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB03-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Tue, 2 Oct 2012 08:38:13 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id q92FcDh48007; Tue, 2 Oct 2012 08:38:13 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id D766B58093; Tue, 2 Oct 2012 08:38:12 -0700 (PDT) To: John Baldwin In-Reply-To: <201210020750.23358.jhb@freebsd.org> References: <20121001223100.E7D0D58093@chaos.jnpr.net> <201210020750.23358.jhb@freebsd.org> Comments: In-reply-to: John Baldwin message dated "Tue, 02 Oct 2012 07:50:23 -0400." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Tue, 2 Oct 2012 08:38:12 -0700 Message-ID: <20121002153812.D766B58093@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-Mailman-Approved-At: Tue, 02 Oct 2012 15:44:31 +0000 Cc: Garrett Cooper , freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org, sjg@juniper.net Subject: Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 15:39:21 -0000 On Tue, 2 Oct 2012 07:50:23 -0400, John Baldwin writes: >BTW, one general comment. There seem to be two completely independent >groups of folks working on ATF (e.g. there have been two different >imports of ATF into the tree in two different locations IIRC, and now >we have two different sets of patches to our system makefiles). Yes, and no. We (Juniper) have been using ATF for some time, and were going to do import etc, but for various reasons haven't done it yet. In part I guess because having bmake in tree would have made things much simpler - avoiding re-inventing wheels. Garrett, has forged ahead, and we are fine with that - Marcel did the import for him. Obviously though (as I've probably just made clear) we don't want ATF to unnecessarily complicate the build. We hope to get the initial bmake commit done this week, and then we can help Garrett get ATF going with minimal fuss. >Are these two groups talking to each other at all? Yes, but I don't think Garrett was aware of the bmake work. > It seems there are some differences in >the two approaches that merit working out to avoid a lot of wasted >effort on both sides. The differences are probably very minor, and hopefully easily sorted out. The most significant being how to build the multiple test apps in one directory. Related to that is the exact location. I believe we are all agreed that tests should be co-located with the code they exercise - to facilitate testing as you make changes. We use a tests/ subdir per lib and prog that has unit-tests and I would recommend doing the same. This is consistent with the above goal, and the separate directory is very useful for keeping the build simple - eg. libfoo/Makefile can continue to just use bsd.lib.mk while libfoo/tests/Makefile can use any of bsd.prog.mk, bsd.progs.mk, bsd.test.mk or atf.test.mk Also, these separate dirs become even more important when using meta mode because you want the all/default target to "just do the right thing", and do it the same way every time, to avoid churn in dependencies. >Do we already have a freebsd-atf@ mailing list? If not, perhaps we >should create one and start these discussions there? Don't know, but fine either way. --sjg From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 2 16:19:03 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7714D106566B; Tue, 2 Oct 2012 16:19:03 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from exprod7og124.obsmtp.com (exprod7og124.obsmtp.com [64.18.2.26]) by mx1.freebsd.org (Postfix) with ESMTP id 0CC958FC15; Tue, 2 Oct 2012 16:19:01 +0000 (UTC) Received: from P-EMHUB03-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob124.postini.com ([64.18.6.12]) with SMTP ID DSNKUGsT71yeK0mQ0bP/uNjtntajTyvZOkML@postini.com; Tue, 02 Oct 2012 09:19:03 PDT Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB03-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Tue, 2 Oct 2012 09:17:40 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id q92GHeh43108; Tue, 2 Oct 2012 09:17:40 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id EFE0E58093; Tue, 2 Oct 2012 09:17:39 -0700 (PDT) To: Garrett Cooper In-Reply-To: References: <20121001223100.E7D0D58093@chaos.jnpr.net> <20121002000030.54CEE58093@chaos.jnpr.net> Comments: In-reply-to: Garrett Cooper message dated "Tue, 02 Oct 2012 07:19:55 -0700." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Tue, 2 Oct 2012 09:17:39 -0700 Message-ID: <20121002161739.EFE0E58093@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-Mailman-Approved-At: Tue, 02 Oct 2012 16:36:28 +0000 Cc: freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org, sjg@juniper.net Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 16:19:03 -0000 On Tue, 2 Oct 2012 07:19:55 -0700, Garrett Cooper writes: >> We put the test cases in a subdir of the lib/prog >> This has multiple benefits, and eliminates any impact on the normal >> build of said libs/progs. > >Hmmm... that's one of the 3 approaches I provided, but it turned out >to be annoying to make this work at first inspection because of how >things were done in our build system. Just for review (even though >it's OT), the three approaches I presented were as follows... > >Note: in all 3 examples, clock.c is the source code and t_clock.c is >the relevant unittest code. There is actually another: lib/libc/Makefile lib/libc/gen/... .../clock.c .../t_clock.c lib/libc/tests/Makefile that is the makefile for building/running the tests lives in the subdir, there are advantages to this, but the test code itself can be anywhere you like. Either next to the code that it tests, or in the tests/ subdir, another subdir, or whatever combination makes the most sense. Most of our ATF users put their test code in the subdir fwiw. >> Also a number of our teams find it necessary to create mock classes etc to >> adequately test their libs. Keeping all this in a tests/ subdir makes >> it easy, to keep things neat & up to date, and ensure that the tests >> pass. Trying to do all that in the same dir as the lib would be ugly. > >This (consolidating everything under a single directory) is the way >that was requested by the beforementioned parties. Sorry for not participating in that discussion ;-) John - perhaps we do need that ATF list? For example building a library and test apps in the one directory either requires turning bsd.lib.mk into an even more complex thing that a multi-prog bsd.pog.mk, or having the makefile behave completely differently depending on the target requested. That later is a bad idea. It precludes being able to seamlessly integrate unit-tests into the build, and would need to be fixed if/when attempting to introduce meta mode. Typically the unit-tests depend on the library they are testing, a separate subdir for the tests/Makefile makes capturing that dependency easy - otherwise you are back to the unspeakably complex bsd.lib.mk Far better to get these things right first up. >I wish I had known that. I pinged marcel/obrien about this some weeks >ago and didn't receive a response. Many appologies. >What are the plans for importing bmake into FreeBSD (this should >probably be a different thread)? The import has been done - just needs to be merged to contrib, and there's a small patch to be committed. We hope to get that done this week. >I agree that it's a good thing (in theory) -- it'd just help to >discuss this with Julio first because I know of a couple cases where >this would be unusable given how bsd.test.mk is coded: I do talk to Julio about ATF (I'm sjg@netbsd.org ;-) though not specifically this - beyond (I think) mentioning that I did it differently. >1. The "it works for me on my machine!" certification: > >It encourages environment tainting, which could be a really, really >bad thing; I've seen developers take interesting shortcuts when True. In our (Junos) build we cleanse the environment rather carefully and have a very clear distinction between building for the "host" (which may just happend to be i386 based) and the i386 "target" for example. I can provide build smarts to do all that - but didn't want to shove it down anyone's throat ;-) What "works for us" doesn't necessarily work for everyone. >2. Failure in a subdirectory results in lost coverage: > >a/... > .../b/... <- tests live here > .../c/d/... <- tests live here > >If I execute make test from a/ (one of my goals), then it should >iterate down a/b, then a/c/d and run the tests in a DFS style (BFS if This gets back to the bmake topic. We actually avoid "walking" the tree at all, eg. in the Junos build I have a target "atf-tests" which builds all ATF tests dirs regardless of where they are (and any pre-requisits they have). This is handy to include as a dependency of the build target we use for branch syncs etc, to ensure no unit-tests atrophy. But the most common case - and the one to optimize for, is the person making changes to libfoo, testing that he hasn't broken anything. >could disguise other bugs. This would be fixed by changing bsd.test.mk >to use atf-run properly, but that's work (not hard) that still needs >to be done. Yes, atf.test.mk already does that. >Hmmmm... My goal was to make ATF a "one ring to rule them all" >infrastructure so that way everything could be unified under ATF in ATF is a pretty good framework - which is why we (Juniper) use it. We want the flexibility to have more than one framework, but the number should be very small. So far I have persuaded teams that wanted to adopt alternate frameworks, that ATF could easily meet their needs. >one way, shape or form (even if it's just reporting), because lord >knows we don't need a lua, unittest/nose, etc wrapper for reporting >results. All of the jobs I've worked at write custom wrappers around Yes, having all "frameworks" able to output results in a consistent format is definitely a good thing. Thanks --sjg From owner-freebsd-hackers@FreeBSD.ORG Tue Oct 2 18:16:49 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 94AFD106564A; Tue, 2 Oct 2012 18:16:49 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 66C558FC12; Tue, 2 Oct 2012 18:16:49 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id C10C6B911; Tue, 2 Oct 2012 14:16:48 -0400 (EDT) From: John Baldwin To: Garrett Cooper Date: Tue, 2 Oct 2012 10:37:27 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p20; KDE/4.5.5; amd64; ; ) References: <201210020750.23358.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201210021037.27762.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 02 Oct 2012 14:16:48 -0400 (EDT) Cc: freebsd-hackers@freebsd.org, "Simon J. Gerraty" , freebsd-arch@freebsd.org Subject: Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 18:16:49 -0000 On Tuesday, October 02, 2012 10:29:49 am Garrett Cooper wrote: > On Tue, Oct 2, 2012 at 4:50 AM, John Baldwin wrote: > > ... > > > This sounds like a superior approach. It doesn't break any current use > > cases while giving the ability to build multiple programs in the few > > places that need it. It sounds like there are a few places under gnu/ > > from Garrett's reply that might be able to make use of this as well. > > For the record, gnu/cc/cc_tools/Makefile is where I first spotted a > potential "bsd.progs.mk" candidate. Most of the other code doesn't > care given how things are organized in our source tree. > > > BTW, one general comment. There seem to be two completely independent > > groups of folks working on ATF (e.g. there have been two different > > imports of ATF into the tree in two different locations IIRC, and now > > we have two different sets of patches to our system makefiles). > > > > Are these two groups talking to each other at all? I know in May that > > many folks (certainly multiple vendors) are interested in ATF, and it > > seems that both Juniper and Isilon have ported ATF internally. It seems > > that it might be good for the two groups to work together to avoid > > stomping on each other's toes. It seems there are some differences in > > the two approaches that merit working out to avoid a lot of wasted > > effort on both sides. > > Both parties (Isilon/Juniper) are converging on the ATF porting work > that Giorgos/myself have done after talking at the FreeBSD Foundation > meet-n-greet. I have contributed all of the patches that I have other > to marcel for feedback. This is very non-obvious to the public at large (e.g. there was no public response to one group's inquiry about the second ATF import for example). Also, given that you had no idea that sgf@ and obrien@ were working on importing NetBSD's bmake as a prerequisite for ATF, it seems that whatever discussions were held were not very detailed at best. I think it would be good to have the various folks working on ATF to at least summarize the current state of things and sketch out some sort of plan or roadmap for future work in a public forum (such as atf@, though a summary mail would be quite appropriate for arch@). -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 3 04:03:17 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B2DE106564A; Wed, 3 Oct 2012 04:03:17 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) by mx1.freebsd.org (Postfix) with ESMTP id DF28F8FC08; Wed, 3 Oct 2012 04:03:16 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.5/8.14.5) with ESMTP id q9343FEk078081; Wed, 3 Oct 2012 00:03:15 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.5/8.14.4/Submit) id q9343FuC078078; Wed, 3 Oct 2012 00:03:15 -0400 (EDT) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <20587.47363.504969.926603@hergotha.csail.mit.edu> Date: Wed, 3 Oct 2012 00:03:15 -0400 From: Garrett Wollman To: Rick Macklem In-Reply-To: <499414315.1544891.1349180909058.JavaMail.root@erie.cs.uoguelph.ca> References: <20586.27582.478147.838896@hergotha.csail.mit.edu> <499414315.1544891.1349180909058.JavaMail.root@erie.cs.uoguelph.ca> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (hergotha.csail.mit.edu [127.0.0.1]); Wed, 03 Oct 2012 00:03:15 -0400 (EDT) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu X-Mailman-Approved-At: Wed, 03 Oct 2012 04:09:04 +0000 Cc: freebsd-fs@freebsd.org, rmacklem@freebsd.org, hackers@freebsd.org Subject: Re: NFS server bottlenecks X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 04:03:17 -0000 [Adding freebsd-fs@ to the Cc list, which I neglected the first time around...] < said: > I can't remember (I am early retired now;-) if I mentioned this patch before: > http://people.freebsd.org/~rmacklem/drc.patch > It adds tunables vfs.nfsd.tcphighwater and vfs.nfsd.udphighwater that can > be twiddled so that the drc is trimmed less frequently. By making these > values larger, the trim will only happen once/sec until the high water > mark is reached, instead of on every RPC. The tradeoff is that the DRC will > become larger, but given memory sizes these days, that may be fine for you. It will be a while before I have another server that isn't in production (it's on my deployment plan, but getting the production servers going is taking first priority). The approaches that I was going to look at: Simplest: only do the cache trim once every N requests (for some reasonable value of N, e.g., 1000). Maybe keep track of the number of entries in each hash bucket and ignore those buckets that only have one entry even if is stale. Simple: just use a sepatate mutex for each list that a cache entry is on, rather than a global lock for everything. This would reduce the mutex contention, but I'm not sure how significantly since I don't have the means to measure it yet. Moderately complicated: figure out if a different synchronization type can safely be used (e.g., rmlock instead of mutex) and do so. More complicated: move all cache trimming to a separate thread and just have the rest of the code wake it up when the cache is getting too big (or just once a second since that's easy to implement). Maybe just move all cache processing to a separate thread. It's pretty clear from the profile that the cache mutex is heavily contended, so anything that reduces the length of time it's held is probably a win. That URL again, for the benefit of people on freebsd-fs who didn't see it on hackers, is: >> . (This graph is slightly modified from my previous post as I removed some spurious edges to make the formatting look better. Still looking for a way to get a profile that includes all kernel modules with the kernel.) -GAWollman From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 3 09:41:53 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2095C106566B for ; Wed, 3 Oct 2012 09:41:53 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84]) by mx1.freebsd.org (Postfix) with ESMTP id C66B18FC0C for ; Wed, 3 Oct 2012 09:41:52 +0000 (UTC) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by kabab.cs.huji.ac.il with esmtp id 1TJLSb-0008k7-Bd for hackers@freebsd.org; Wed, 03 Oct 2012 11:41:45 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.3 To: hackers@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 03 Oct 2012 11:41:45 +0200 From: Daniel Braniss Message-ID: Cc: Subject: problem cross-compiling 9.1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 09:41:53 -0000 reposting to hackers, maybe better luck here? When using an amd64 host to 'make TARGET_ARCH=i386 buildworld' it seems that it's using the wrong cpp, at least when building ioctl.c via mkioctl in usr.bin/ktrace and having set WITHOUT_CPP(*). This used to work with previous releases. ... ===> usr.bin/kdump (depend) env CPP="cpp" sh /r+d/stable/9/usr.bin/kdump/mkioctls /home/obj/rnd/alix/i386.i386/r+d/stable/9/tmp/usr/include > ioctl.c ... cc -O2 -pipe -I/r+d/stable/9/usr.bin/kdump/../ktrace -I/r+d/stable/9/usr.bin/kdump -I/r+d/stable/9/usr.bin/kdump/../.. -DNDEBUG -std=gnu99 -fstack-protector -Wno-pointer-sign -c ioctl.c ioctl.c: In function 'ioctlname': ioctl.c:1216: error: 'MPTIO_RAID_ACTION32' undeclared (first use in this function) ioctl.c:1216: error: (Each undeclared identifier is reported only once ioctl.c:1216: error: for each function it appears in.) ioctl.c:1292: error: 'MPTIO_READ_EXT_CFG_HEADER32' undeclared (first use in this function) ioctl.c:1632: error: 'MPTIO_READ_EXT_CFG_PAGE32' undeclared (first use in this function) ioctl.c:1772: error: 'CCISS_PASSTHRU32' undeclared (first use in this function) ioctl.c:2010: error: 'IPMICTL_RECEIVE_MSG_TRUNC_32' undeclared (first use in this function) ioctl.c:2082: error: 'IPMICTL_RECEIVE_MSG_32' undeclared (first use in this function) ioctl.c:2300: error: 'MPTIO_READ_CFG_PAGE32' undeclared (first use in this function) ioctl.c:2870: error: 'MPTIO_READ_CFG_HEADER32' undeclared (first use in this function) ioctl.c:2878: error: 'IPMICTL_SEND_COMMAND_32' undeclared (first use in this function) ioctl.c:2938: error: 'MPTIO_WRITE_CFG_PAGE32' undeclared (first use in this function) *** [ioctl.o] Error code 1 *: Im compiling for an embedded system, and hence I want the minimum stuff. at the moment the work around is to remove the WITHOUT_CPP, but it got me worried. any fix? cheers, danny From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 3 11:23:40 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2BD0E1065672 for ; Wed, 3 Oct 2012 11:23:40 +0000 (UTC) (envelope-from ramquick@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id B32F78FC15 for ; Wed, 3 Oct 2012 11:23:39 +0000 (UTC) Received: by eekc50 with SMTP id c50so4168331eek.13 for ; Wed, 03 Oct 2012 04:23:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:date:message-id:subject:from:to:content-type; bh=kHSynCOu98vNI3TA7r+Ijzpnmca2P6p+zmH8SJKweA4=; b=Se3TiaoG03erdQoJXtiS7gNO+V3UA5IADEOIr1sDNNiIOlN7WbI3g7ah2H0Vpj3YLc 8Bv9FqojVJ39w4gAmaQrXKfLqMXJilgp06dL1KiivalzVqxuSnLa8tiPZC0j2nu+4QNk Qd0LzgjlRVRyX2H5ZcPr7VNW45xhZLp6wOn9fVv5EoW5XTpPpIB/NIvkVUqVOGhLM8zT XM3Jp1NrzoAqq1KDGv0wl4dljxPMGbOAW74wYvZv/j5WdfKMoq8rTjwI8vn+PPeXRUW8 Z6JjANfjpz7jNFlrR2RhEiLAmSDQ8fWZGmdrvNJ0/7r5hV2qZEoSroE71SKYwlXQfEEI TszQ== MIME-Version: 1.0 Received: by 10.14.209.5 with SMTP id r5mr2149103eeo.28.1349263413556; Wed, 03 Oct 2012 04:23:33 -0700 (PDT) Received: by 10.14.198.4 with HTTP; Wed, 3 Oct 2012 04:23:33 -0700 (PDT) Date: Wed, 3 Oct 2012 16:53:33 +0530 Message-ID: From: Ram Chander To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Zfs import issue X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ram_chander250@yahoo.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 11:23:40 -0000 Hi, I am importing zfs snapshot to freebsd-9 from anther host running freebsd-9. When the import happens, it locks the filesystem, "df" hangs and unable to use the filesystem. Once the import completes, the filesystem is back to normal and read/write works fine. The same doesnt happen in Solaris/OpenIndiana. # uname -an FreeBSD hostname 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan 3 07:46:30 UTC 2012 root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 Zfs ver: 28 Any inputs would be helpful. Is there any way to overcome this freeze ? Regards, Ram From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 3 13:21:08 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 83CA2106566B; Wed, 3 Oct 2012 13:21:08 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id E31C48FC14; Wed, 3 Oct 2012 13:21:07 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAHk6bFCDaFvO/2dsb2JhbABFFoV3uW6CIAEBAQMBAQEBICsgCwUWGAICDRkCKQEJJgYIBwQBHAEDh14GC6UnkmSBIYoCGoUOgRIDkzyCLYEVjxaDCYFHNA X-IronPort-AV: E=Sophos;i="4.80,528,1344225600"; d="scan'208";a="181737325" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 03 Oct 2012 09:21:06 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 85A10B4032; Wed, 3 Oct 2012 09:21:06 -0400 (EDT) Date: Wed, 3 Oct 2012 09:21:06 -0400 (EDT) From: Rick Macklem To: Garrett Wollman Message-ID: <1571646304.1630985.1349270466529.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20587.47363.504969.926603@hergotha.csail.mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org, rmacklem@freebsd.org, hackers@freebsd.org Subject: Re: NFS server bottlenecks X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 13:21:08 -0000 Garrett Wollman wrote: > [Adding freebsd-fs@ to the Cc list, which I neglected the first time > around...] > > < said: > > > I can't remember (I am early retired now;-) if I mentioned this > > patch before: > > http://people.freebsd.org/~rmacklem/drc.patch > > It adds tunables vfs.nfsd.tcphighwater and vfs.nfsd.udphighwater > > that can > > be twiddled so that the drc is trimmed less frequently. By making > > these > > values larger, the trim will only happen once/sec until the high > > water > > mark is reached, instead of on every RPC. The tradeoff is that the > > DRC will > > become larger, but given memory sizes these days, that may be fine > > for you. > > It will be a while before I have another server that isn't in > production (it's on my deployment plan, but getting the production > servers going is taking first priority). > > The approaches that I was going to look at: > > Simplest: only do the cache trim once every N requests (for some > reasonable value of N, e.g., 1000). Maybe keep track of the number of > entries in each hash bucket and ignore those buckets that only have > one entry even if is stale. > Well, the patch I have does it when it gets "too big". This made sense to me, since the cache is trimmed to keep it from getting too large. It also does the trim at least once/sec, so that really stale entries are removed. > Simple: just use a sepatate mutex for each list that a cache entry > is on, rather than a global lock for everything. This would reduce > the mutex contention, but I'm not sure how significantly since I > don't have the means to measure it yet. > Well, since the cache trimming is removing entries from the lists, I don't see how that can be done with a global lock for list updates? A mutex in each element could be used for changes (not insertion/removal) to an individual element. However, the current code manipulates the lists and makes minimal changes to the individual elements, so I'm not sure if a mutex in each element would be useful or not, but it wouldn't help for the trimming case, imho. I modified the patch slightly, so it doesn't bother to acquire the mutex when it is checking if it should trim now. I think this results in a slight risk that the test will use an "out of date" cached copy of one of the global vars, but since the code isn't modifying them, I don't think it matters. This modified patch is attached and is also here: http://people.freebsd.org/~rmacklem/drc2.patch > Moderately complicated: figure out if a different synchronization type > can safely be used (e.g., rmlock instead of mutex) and do so. > > More complicated: move all cache trimming to a separate thread and > just have the rest of the code wake it up when the cache is getting > too big (or just once a second since that's easy to implement). Maybe > just move all cache processing to a separate thread. > Only doing it once/sec would result in a very large cache when bursts of traffic arrives. The above patch does it when it is "too big" or at least once/sec. I'm not sure I see why doing it as a separate thread will improve things. There are N nfsd threads already (N can be bumped up to 256 if you wish) and having a bunch more "cache trimming threads" would just increase contention, wouldn't it? The only negative effect I can think of w.r.t. having the nfsd threads doing it would be a (I believe negligible) increase in RPC response times (the time the nfsd thread spends trimming the cache). As noted, I think this time would be negligible compared to disk I/O and network transit times in the total RPC response time? Isilon did use separate threads (I never saw their code, so I am going by what they told me), but it sounded to me like they were trimming the cache too agressively to be effective for TCP mounts. (ie. It sounded to me like they had broken the algorithm to achieve better perf.) Remember that the DRC is weird, in that it is a cache to improve correctness at the expense of overhead. It never improves performance. On the other hand, turn it off or throw away entries too aggressively and data corruption, due to retries of non-idempotent operations, can be the outcome. Good luck with whatever you choose, rick > It's pretty clear from the profile that the cache mutex is heavily > contended, so anything that reduces the length of time it's held is > probably a win. > > That URL again, for the benefit of people on freebsd-fs who didn't see > it on hackers, is: > > >> . > > (This graph is slightly modified from my previous post as I removed > some spurious edges to make the formatting look better. Still looking > for a way to get a profile that includes all kernel modules with the > kernel.) > > -GAWollman > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 3 14:05:22 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3C311106566B; Wed, 3 Oct 2012 14:05:22 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 0C6828FC17; Wed, 3 Oct 2012 14:05:20 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA06821; Wed, 03 Oct 2012 17:05:19 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <506C461F.2050008@FreeBSD.org> Date: Wed, 03 Oct 2012 17:05:19 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120911 Thunderbird/15.0.1 MIME-Version: 1.0 To: attilio@FreeBSD.org References: <50587F8D.9060102@FreeBSD.org> <5058C68B.1010508@FreeBSD.org> <50596019.5060708@FreeBSD.org> <505AF836.7050004@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-hackers , Jeff Roberson Subject: Re: ule+smp: small optimization for turnstile priority lending X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 14:05:22 -0000 on 20/09/2012 16:14 Attilio Rao said the following: > On 9/20/12, Andriy Gapon wrote: [snip] >> The patch works well as far as I can tell. Thank you! >> There is one warning with full witness enables but it appears to be harmless >> (so >> far): > > Andriy, > thanks a lot for your testing and reports you made so far. > Unfortunately I'm going off for 2 weeks now and I won't work on > FreeBSD for that timeframe. I will get back to those in 2 weeks then. > If you want to play more with this idea feel free to extend/fix/etc. > this patch. Unfortunately I haven't found time to work on this further, but I have some additional thoughts. First, I think that the witness message was not benign and it actually warns about the same kind of deadlock that I originally had. The problem is that sched_lend_prio() is called with target thread's td_lock held, which is a lock of tdq on the thread's CPU. Then, in your patch, we acquire current tdq's lock to modify its load. But if two tdq locks are to be held at the same time, then they must be locked using tdq_lock_pair, so that lock order is maintained. With the patch no tdq lock order can be maintained (can arbitrary) and thus it is possible to run into a deadlock. I see two possibilities here, but don't rule out that there can be more. 1. Do something like my patch does. That is, manipulate current thread's tdq in advance before any other thread/tdq locks are acquired (or td_lock is changed to point to a different lock and current tdq is unlocked). The API can be made more generic in nature. E.g. it can look like this: void sched_thread_block(struct thread *td, int inhibitor) { struct tdq *tdq; THREAD_LOCK_ASSERT(td, MA_OWNED); KASSERT(td == curthread, ("sched_thread_block: only curthread is supported")); tdq = TDQ_SELF(); TDQ_LOCK_ASSERT(tdq, MA_OWNED); MPASS(td->td_lock == TDQ_LOCKPTR(tdq)); TD_SET_INHIB(td, inhibitor); tdq_load_rem(tdq, td); tdq_setlowpri(tdq, td); } 2. Try to do things from sched_lend_prio based on curthread's state. This, as it seems, would require completely lock-less manipulations of current tdq. E.g. for tdq_load we could use atomic operations (it is already accessed locklessly, but not modified). But for tdq_lowpri a more elaborate trick would be required like having a separate field for a temporary value. In any case, I'll have to revisit this later. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 3 15:16:14 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 383BF106564A for ; Wed, 3 Oct 2012 15:16:14 +0000 (UTC) (envelope-from willingbug@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id EF4838FC08 for ; Wed, 3 Oct 2012 15:16:13 +0000 (UTC) Received: by obbwc20 with SMTP id wc20so8395789obb.13 for ; Wed, 03 Oct 2012 08:16:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=5vMDZHcJMP+hZokERvw8+Y4YgPUovX8DK+wUG1pa/R4=; b=eNZV2yel4uRhGNHiw4ow6vHgIJLlk07Ex6BopW24qMEvh03e5yBvt8YQUrh7hFwv1Z GEMiX5LKArHmXHkQu59tWBp+K0yee5LpUmEJ+nLQgdeE7rom+GqpOBtiUMVsEPIR456p cI9finVChmyGyBXAEx7t3eOz4MUQf4BfgWMnsPzwculN41TK17fx2/tAokwsJc6IE8NK l0VPje6Zf6EW0o/dgTXFVy6eVGemOmPHIDGxrWrt0o+A2/+vP0PKHxD4O1Pl2/CfyKOM bk3EnY8OB7JMNUSJK3+sLOSIBy7Z+wmutY9E8RF28sNmcWKe361YqVaRXxyjzaEL1hRw tPkQ== MIME-Version: 1.0 Received: by 10.60.22.196 with SMTP id g4mr1801670oef.95.1349277373268; Wed, 03 Oct 2012 08:16:13 -0700 (PDT) Received: by 10.182.49.130 with HTTP; Wed, 3 Oct 2012 08:16:13 -0700 (PDT) Date: Wed, 3 Oct 2012 23:16:13 +0800 Message-ID: From: cz li To: freebsd-hackers Content-Type: text/plain; charset=UTF-8 Subject: help me,my virtualbox run error. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 15:16:14 -0000 my os version is freebsd 9.0.I installed virtualbox 4.0.2.Installation no error.but, can not run.error infromation: "Failed to create the VirtualBox COM object. The application will now terminate. Callee RC: NS_ERROR_FACTORY_NOT_REGISTERED (0x80040154) " How to solve this problem? Thank you. Eagerly look forward to your reply. lichaozhong 2012-10-3 From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 3 15:27:20 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C723F106564A for ; Wed, 3 Oct 2012 15:27:20 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 785328FC0C for ; Wed, 3 Oct 2012 15:27:20 +0000 (UTC) Received: by vbmv11 with SMTP id v11so10395655vbm.13 for ; Wed, 03 Oct 2012 08:27:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=h6nJC+gtIHWLInmPI8Rp4fXtucgVf1AANgA5F+xar78=; b=RVOaxTfNHAWvnULjwBOrrQfIu+uKMtZSoYTMlCjliVCIkKbVUkrLguY/P3jfaJVOxC SvFzSKxDWmM3+puyH/8BhJPinBBd2bBcAi5a3J0K7fky8mFpLsw0YVH4cAVugCD2FovV IQJaC+LPkR3CkGVxli7DoBniS+GNV2G8UK8xGt4HVI2HJjx5LPaTs56QSRkY20nVmQd6 QVu7cabNPB7vwM9A9V5TX/1LpXQmAoeIwglcpCIFGdVUcp84qIJu48qwEkSJdvK+mUXD HSQw3FriLijNkbHkr7nd/UpB0jjORt58gE6Q/GXB/2X5rO86eZ9o18JGUqgmtQFVZxHE G/UA== MIME-Version: 1.0 Received: by 10.220.227.199 with SMTP id jb7mr1320020vcb.26.1349278039594; Wed, 03 Oct 2012 08:27:19 -0700 (PDT) Received: by 10.58.69.33 with HTTP; Wed, 3 Oct 2012 08:27:19 -0700 (PDT) In-Reply-To: References: Date: Wed, 3 Oct 2012 16:27:19 +0100 Message-ID: From: Tom Evans To: cz li Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers Subject: Re: help me,my virtualbox run error. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 15:27:20 -0000 On Wed, Oct 3, 2012 at 4:16 PM, cz li wrote: > my os version is freebsd 9.0.I installed virtualbox 4.0.2.Installation > no error.but, can not run.error infromation: > "Failed to create the VirtualBox COM object. > > The application will now terminate. > > > > Callee RC: NS_ERROR_FACTORY_NOT_REGISTERED (0x80040154) > " https://www.virtualbox.org/ticket/2335 Cheers Tom From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 3 16:37:09 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4D1E71065672 for ; Wed, 3 Oct 2012 16:37:09 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 9FA058FC12 for ; Wed, 3 Oct 2012 16:37:08 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA08079 for ; Wed, 03 Oct 2012 19:37:07 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <506C69B3.5060104@FreeBSD.org> Date: Wed, 03 Oct 2012 19:37:07 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120911 Thunderbird/15.0.1 MIME-Version: 1.0 To: freebsd-hackers X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: kvm_proclist: gnore processes in PRS_NEW X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 16:37:09 -0000 I believe that the following patch does the right thing that is repeated in a few other places. I would like to ask for a review just in case. commit cf0f573a1dcbc09cb8fce612530afeeb7f1b1c62 Author: Andriy Gapon Date: Sun Sep 23 22:49:26 2012 +0300 kvm_proc: ignore processes in larvae state diff --git a/lib/libkvm/kvm_proc.c b/lib/libkvm/kvm_proc.c index 8fc415c..d1daf77 100644 --- a/lib/libkvm/kvm_proc.c +++ b/lib/libkvm/kvm_proc.c @@ -144,6 +144,8 @@ kvm_proclist(kvm_t *kd, int what, int arg, struct proc *p, _kvm_err(kd, kd->program, "can't read proc at %p", p); return (-1); } + if (proc.p_state == PRS_NEW) + continue; if (proc.p_state != PRS_ZOMBIE) { if (KREAD(kd, (u_long)TAILQ_FIRST(&proc.p_threads), &mtd)) { -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 3 16:57:56 2012 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D88C7106566C for ; Wed, 3 Oct 2012 16:57:56 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 3642B8FC0C for ; Wed, 3 Oct 2012 16:57:55 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA08280 for ; Wed, 03 Oct 2012 19:57:54 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <506C6E92.10803@FreeBSD.org> Date: Wed, 03 Oct 2012 19:57:54 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120911 Thunderbird/15.0.1 MIME-Version: 1.0 To: freebsd-hackers X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: kvm_getprocs: gracefully handle errors in kvm_deadprocs X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 16:57:56 -0000 kvm_deadprocs returns -1 to signify an error. Current kvm_getprocs code would pass this return code as 'cnt' out parameter and would not reset return value to NULL. This confuses some callers, most prominently procstat_getprocs, into believing that kvm_getprocs was successful. Moreover, the code tried to use cnt=-1 as an unsigned number to allocate some additional memory. As a result fstat -M could try to allocate huge amount of memory e.g. when used with a kernel that didn't match userland. With the proposed change the error code should be handled properly. Additionally it should now be possible to enable realloc code, which previously contained a bug and was called even after kvm_deadprocs error. commit 6ddf602409119eded40321e5bb349b464f24e81a Author: Andriy Gapon Date: Sun Sep 23 22:52:28 2012 +0300 kvm_proc: gracefully handle errors in kvm_deadprocs, don't confuse callers Plus fix a bug under 'notdef' (sic) section. diff --git a/lib/libkvm/kvm_proc.c b/lib/libkvm/kvm_proc.c index d1daf77..31258d7 100644 --- a/lib/libkvm/kvm_proc.c +++ b/lib/libkvm/kvm_proc.c @@ -593,9 +593,15 @@ liveout: nprocs = kvm_deadprocs(kd, op, arg, nl[1].n_value, nl[2].n_value, nprocs); + if (nprocs <= 0) { + _kvm_freeprocs(kd); + nprocs = 0; + } #ifdef notdef - size = nprocs * sizeof(struct kinfo_proc); - (void)realloc(kd->procbase, size); + else { + size = nprocs * sizeof(struct kinfo_proc); + kd->procbase = realloc(kd->procbase, size); + } #endif } *cnt = nprocs; P.S. it might may sense to change 'count' parameter of procstat_getprocs to signed int so that it matches kvm_getprocs interface. Alternatively, an intermediate variable could be used to insulate signedness difference: index 56562e1..11a817e 100644 --- a/lib/libprocstat/libprocstat.c +++ b/lib/libprocstat/libprocstat.c @@ -184,15 +184,18 @@ procstat_getprocs(struct procstat *procstat, int what, int arg, struct kinfo_proc *p0, *p; size_t len; int name[4]; + int cnt; int error; assert(procstat); assert(count); p = NULL; if (procstat->type == PROCSTAT_KVM) { - p0 = kvm_getprocs(procstat->kd, what, arg, count); - if (p0 == NULL || count == 0) + *count = 0; + p0 = kvm_getprocs(procstat->kd, what, arg, &cnt); + if (p0 == NULL || cnt <= 0) return (NULL); + *count = cnt; len = *count * sizeof(*p); p = malloc(len); if (p == NULL) { -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 3 17:37:54 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8EFB71065672 for ; Wed, 3 Oct 2012 17:37:54 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from ns1.jnielsen.net (secure.freebsdsolutions.net [69.55.234.48]) by mx1.freebsd.org (Postfix) with ESMTP id 52B978FC0C for ; Wed, 3 Oct 2012 17:37:53 +0000 (UTC) Received: from [10.10.1.32] (office.betterlinux.com [199.58.199.60]) (authenticated bits=0) by ns1.jnielsen.net (8.14.4/8.14.4) with ESMTP id q93HbaLT035313 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Wed, 3 Oct 2012 13:37:36 -0400 (EDT) (envelope-from lists@jnielsen.net) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) From: John Nielsen In-Reply-To: <20121002083634.3103fe958508a4026384ac96@yamagi.org> Date: Wed, 3 Oct 2012 11:37:39 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5069C9FC.6020400@brandonfa.lk> <87549776-9051-4B4B-8D53-DAE6D51C2A94@kientzle.com> <20121002083634.3103fe958508a4026384ac96@yamagi.org> To: Yamagi Burmeister X-Mailer: Apple Mail (2.1498) X-DCC-x.dcc-servers-Metrics: ns1.jnielsen.net 104; Body=4 Fuz1=4 Fuz2=4 X-Virus-Scanned: clamav-milter 0.97.5 at ns1.jnielsen.net X-Virus-Status: Clean Cc: bfalk_bsd@brandonfa.lk, freebsd-hackers@freebsd.org Subject: Re: SMP Version of tar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 17:37:54 -0000 On Oct 2, 2012, at 12:36 AM, Yamagi Burmeister wrote: > On Mon, 1 Oct 2012 22:16:53 -0700 > Tim Kientzle wrote: >=20 >> There are a few different parallel command-line compressors and = decompressors in ports; experiment a lot (with large files being read = from and/or written to disk) and see what the real effect is. In = particular, some decompression algorithms are actually faster than = memcpy() when run on a single processor. Parallelizing such algorithms = is not likely to help much in the real world. >>=20 >> The two popular algorithms I would expect to benefit most are bzip2 = compression and lzma compression (targeting xz or lzip format). For = decompression, bzip2 is block-oriented so fits SMP pretty naturally. = Other popular algorithms are stream-oriented and less amenable to = parallelization. >>=20 >> Take a careful look at pbzip2, which is a parallelized bzip2/bunzip2 = implementation that's already under a BSD license. You should be able = to get a lot of ideas about how to implement a parallel compression = algorithm. Better yet, you might be able to reuse a lot of the existing = pbzip2 code. >>=20 >> Mark Adler's pigz is also worth studying. It's also = license-friendly, and is built on top of regular zlib, which is a nice = technique when it's feasible. >=20 > Just a small note: There's a parallel implementation of xz called > "pixz". It's build atop of liblzma and libarchiv and stands under a=20 > BSD style license. See: https://github.com/vasi/pixz Maybe it's > possible to reuse most of the code. See also below, which has some bugfixes/improvements that AFAIK were = never committed in the original project (though they were submitted). https://github.com/jlrobins/pixz JN From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 3 18:41:14 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BFB8C106566B for ; Wed, 3 Oct 2012 18:41:14 +0000 (UTC) (envelope-from ryao@gentoo.org) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by mx1.freebsd.org (Postfix) with ESMTP id 99BE68FC0C for ; Wed, 3 Oct 2012 18:41:14 +0000 (UTC) Received: from [192.168.1.2] (pool-72-89-250-138.nycmny.fios.verizon.net [72.89.250.138]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: ryao) by smtp.gentoo.org (Postfix) with ESMTPSA id BBF8333C28D for ; Wed, 3 Oct 2012 18:41:12 +0000 (UTC) Message-ID: <506C866C.3050509@gentoo.org> Date: Wed, 03 Oct 2012 14:39:40 -0400 From: Richard Yao User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.7) Gecko/20120917 Thunderbird/10.0.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <5069C9FC.6020400@brandonfa.lk> <87549776-9051-4B4B-8D53-DAE6D51C2A94@kientzle.com> <20121002083634.3103fe958508a4026384ac96@yamagi.org> In-Reply-To: X-Enigmail-Version: 1.3.5 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigE37D80F93A1F4426230E160A" Subject: Re: SMP Version of tar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 18:41:14 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigE37D80F93A1F4426230E160A Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 10/02/2012 03:06 AM, Adrian Chadd wrote: > .. please keep in mind that embedded platforms (a) don't necessarily > benefit from it, and (b) have a very small footprint. Bloating out the > compression/archival tools for the sake of possible SMP support will > make me very, very sad. >=20 >=20 >=20 > Adrian > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" Someone might want to ask if parallelizing tar is even possible. tar is meant to serially write to tape. It should not be possible possible to parallelize that operation. I can imagine parallelizing compression algorithms, but I cannot imagine parallelizing tar. --------------enigE37D80F93A1F4426230E160A Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJQbIZyAAoJECDuEZm+6Exk7isQAJEDfEZtmqYDYXQ3jm1EPRj9 m2yE+rK9oqTfOL4ifzQ85qJg+0AMaK7Oj5ZEN7yzXl4zjHKDG0CG5cM77lPQmGIs FAkn6v1lp2FUfqC6ooDjZmspMHoYAbr7rgLblqr4YEk7NaQOiNhg9kSGnD2YuRVE lwqF6oGw69LqcOUZ85mHiWeMgP3crZGrUkb8r4BLBNWF4rKK+frmnqoRYXcAqKs2 yhaf6Yz4R0xwUdAQc1Vp6nDzqrpvA053FCCYqfvky3CNdpIbbrzDhQ42vmrUprD+ YDvryOjCWfbuNaiECb6IqJY12JIBF+BSwYBvuXmPX/0B8qKbBrbN5bCGC7pc/4SE wewHDzWgV7nZ15CTMbpmk7LduLRtiMpjS9fy6Iwpd8l7qBb9CUt+05K3vnq+cwmp uwiKg79jFsK0ZgH9NK0hqsGZrCNzr+KWp0wfBrtcx8uHoLSlduaZMCnGyQDlZomb Y1PS9zUR16fScqUTk/I7Fbn+vSOcmkWar0nsKxEDaZ+eNfdETG1Sqdf8fg7j6QYn Uv0tE2f+D0e2Gzy6VLa7VLVDnllxt0ftR2PKqweM8usG3YrrptCC4QJbFdahCguY kANKI4bRIXw1Y8/hs9ztS3Ly7QGkYep5USDh4n6EfQXBZeaG0I8beRl5+Sqnit9+ +8UvpYBIGayyy+Y92bH6 =m1kV -----END PGP SIGNATURE----- --------------enigE37D80F93A1F4426230E160A-- From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 3 20:43:14 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 936C0106566C; Wed, 3 Oct 2012 20:43:14 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 6B4068FC0A; Wed, 3 Oct 2012 20:43:14 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id CA026B924; Wed, 3 Oct 2012 16:43:13 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Wed, 3 Oct 2012 15:19:23 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p20; KDE/4.5.5; amd64; ; ) References: <506C69B3.5060104@FreeBSD.org> In-Reply-To: <506C69B3.5060104@FreeBSD.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201210031519.23146.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 03 Oct 2012 16:43:13 -0400 (EDT) Cc: Andriy Gapon Subject: Re: kvm_proclist: gnore processes in PRS_NEW X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 20:43:14 -0000 On Wednesday, October 03, 2012 12:37:07 pm Andriy Gapon wrote: > > I believe that the following patch does the right thing that is repeated in a few > other places. > I would like to ask for a review just in case. > > commit cf0f573a1dcbc09cb8fce612530afeeb7f1b1c62 > Author: Andriy Gapon > Date: Sun Sep 23 22:49:26 2012 +0300 > > kvm_proc: ignore processes in larvae state I think this is fine. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 3 20:59:17 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D5891106564A; Wed, 3 Oct 2012 20:59:17 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 802858FC1C; Wed, 3 Oct 2012 20:59:17 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.5/8.14.5) with ESMTP id q93KxGTY061139; Wed, 3 Oct 2012 16:59:16 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.5/8.14.4/Submit) id q93KxG4D061136; Wed, 3 Oct 2012 16:59:16 -0400 (EDT) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <20588.42788.103863.179701@hergotha.csail.mit.edu> Date: Wed, 3 Oct 2012 16:59:16 -0400 From: Garrett Wollman To: Rick Macklem In-Reply-To: <1571646304.1630985.1349270466529.JavaMail.root@erie.cs.uoguelph.ca> References: <20587.47363.504969.926603@hergotha.csail.mit.edu> <1571646304.1630985.1349270466529.JavaMail.root@erie.cs.uoguelph.ca> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (hergotha.csail.mit.edu [127.0.0.1]); Wed, 03 Oct 2012 16:59:16 -0400 (EDT) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: freebsd-fs@freebsd.org, rmacklem@freebsd.org, hackers@freebsd.org Subject: Re: NFS server bottlenecks X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 20:59:18 -0000 < said: >> Simple: just use a sepatate mutex for each list that a cache entry >> is on, rather than a global lock for everything. This would reduce >> the mutex contention, but I'm not sure how significantly since I >> don't have the means to measure it yet. >> > Well, since the cache trimming is removing entries from the lists, I don't > see how that can be done with a global lock for list updates? Well, the global lock is what we have now, but the cache trimming process only looks at one list at a time, so not locking the list that isn't being iterated over probably wouldn't hurt, unless there's some mechanism (that I didn't see) for entries to move from one list to another. Note that I'm considering each hash bucket a separate "list". (One issue to worry about in that case would be cache-line contention in the array of hash buckets; perhaps NFSRVCACHE_HASHSIZE ought to be increased to reduce that.) > Only doing it once/sec would result in a very large cache when bursts of > traffic arrives. My servers have 96 GB of memory so that's not a big deal for me. > I'm not sure I see why doing it as a separate thread will improve things. > There are N nfsd threads already (N can be bumped up to 256 if you wish) > and having a bunch more "cache trimming threads" would just increase > contention, wouldn't it? Only one cache-trimming thread. The cache trim holds the (global) mutex for much longer than any individual nfsd service thread has any need to, and having N threads doing that in parallel is why it's so heavily contended. If there's only one thread doing the trim, then the nfsd service threads aren't spending time either contending on the mutex (it will be held less frequently and for shorter periods). > The only negative effect I can think of w.r.t. having the nfsd > threads doing it would be a (I believe negligible) increase in RPC > response times (the time the nfsd thread spends trimming the cache). > As noted, I think this time would be negligible compared to disk I/O > and network transit times in the total RPC response time? With adaptive mutexes, many CPUs, lots of in-memory cache, and 10G network connectivity, spinning on a contended mutex takes a significant amount of CPU time. (For the current design of the NFS server, it may actually be a win to turn off adaptive mutexes -- I should give that a try once I'm able to do more testing.) -GAWollman From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 3 21:37:17 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 22F30106564A; Wed, 3 Oct 2012 21:37:17 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 727FE8FC0A; Wed, 3 Oct 2012 21:37:15 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap8EAOCvbFCDaFvO/2dsb2JhbABFhg+5d4IgAQEBAwEBAQEgKyALGxgCAg0ZAikBCSYGCAcEARwBA4deBgulMZJhgSGKAhqFDoESA5M8gi2BFY8WgwmBRzQ X-IronPort-AV: E=Sophos;i="4.80,530,1344225600"; d="scan'208";a="181843957" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 03 Oct 2012 17:36:59 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 34DE3B403A; Wed, 3 Oct 2012 17:36:59 -0400 (EDT) Date: Wed, 3 Oct 2012 17:36:59 -0400 (EDT) From: Rick Macklem To: Garrett Wollman Message-ID: <1666343702.1682678.1349300219198.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20588.42788.103863.179701@hergotha.csail.mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org, rmacklem@freebsd.org, hackers@freebsd.org Subject: Re: NFS server bottlenecks X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 21:37:17 -0000 Garrett Wollman wrote: > < said: > > >> Simple: just use a sepatate mutex for each list that a cache entry > >> is on, rather than a global lock for everything. This would reduce > >> the mutex contention, but I'm not sure how significantly since I > >> don't have the means to measure it yet. > >> > > Well, since the cache trimming is removing entries from the lists, I > > don't > > see how that can be done with a global lock for list updates? > > Well, the global lock is what we have now, but the cache trimming > process only looks at one list at a time, so not locking the list that > isn't being iterated over probably wouldn't hurt, unless there's some > mechanism (that I didn't see) for entries to move from one list to > another. Note that I'm considering each hash bucket a separate > "list". (One issue to worry about in that case would be cache-line > contention in the array of hash buckets; perhaps NFSRVCACHE_HASHSIZE > ought to be increased to reduce that.) > Yea, a separate mutex for each hash list might help. There is also the LRU list that all entries end up on, that gets used by the trimming code. (I think? I wrote this stuff about 8 years ago, so I haven't looked at it in a while.) Also, increasing the hash table size is probably a good idea, especially if you reduce how aggressively the cache is trimmed. > > Only doing it once/sec would result in a very large cache when > > bursts of > > traffic arrives. > > My servers have 96 GB of memory so that's not a big deal for me. > This code was originally "production tested" on a server with 1Gbyte, so times have changed a bit;-) > > I'm not sure I see why doing it as a separate thread will improve > > things. > > There are N nfsd threads already (N can be bumped up to 256 if you > > wish) > > and having a bunch more "cache trimming threads" would just increase > > contention, wouldn't it? > > Only one cache-trimming thread. The cache trim holds the (global) > mutex for much longer than any individual nfsd service thread has any > need to, and having N threads doing that in parallel is why it's so > heavily contended. If there's only one thread doing the trim, then > the nfsd service threads aren't spending time either contending on the > mutex (it will be held less frequently and for shorter periods). > I think the little drc2.patch which will keep the nfsd threads from acquiring the mutex and doing the trimming most of the time, might be sufficient. I still don't see why a separate trimming thread will be an advantage. I'd also be worried that the one cache trimming thread won't get the job done soon enough. When I did production testing on a 1Gbyte server that saw a peak load of about 100RPCs/sec, it was necessary to trim aggressively. (Although I'd be tempted to say that a server with 1Gbyte is no longer relevant, I recently recall someone trying to run FreeBSD on a i486, although I doubt they wanted to run the nfsd on it.) > > The only negative effect I can think of w.r.t. having the nfsd > > threads doing it would be a (I believe negligible) increase in RPC > > response times (the time the nfsd thread spends trimming the cache). > > As noted, I think this time would be negligible compared to disk I/O > > and network transit times in the total RPC response time? > > With adaptive mutexes, many CPUs, lots of in-memory cache, and 10G > network connectivity, spinning on a contended mutex takes a > significant amount of CPU time. (For the current design of the NFS > server, it may actually be a win to turn off adaptive mutexes -- I > should give that a try once I'm able to do more testing.) > Have fun with it. Let me know when you have what you think is a good patch. rick > -GAWollman > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 4 04:11:32 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 72D93106566B for ; Thu, 4 Oct 2012 04:11:32 +0000 (UTC) (envelope-from willingbug@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3232B8FC08 for ; Thu, 4 Oct 2012 04:11:31 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id wc20so84245obb.13 for ; Wed, 03 Oct 2012 21:11:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=EJRuCQqG6ueKt8UhCcmS1Vq8URuGCzyBZ2rFf0P0bfg=; b=xF33An+qi2PUtHBg00+ZHgG2qz3W62eflL6ES7nnxm9SqtKK9dfyytFiuknuNdWmR4 ubBkdupzB2xWO5xp/P9F/EdwH2Fhw1dMDHppZ9De9tBkvqFk2S87237OZz5vbZk9med7 yyWJ3FDd2U7oShRy9PdFmqFY2UdoikiCwaXN8wImVEDMs4p386tnBf7eBX+0zn3PrULp LqUvCCrlQXlXY/GFAINwgtyNKlkJrJdHTeR/a0jKKLn6t/XVVg8PCaYAx+/R5b3gBzsW 9nVGlhaOvxpYuJmKU13Y8X28HveLV8IbCuYIeS/JeJ2e4nPvw+2K5ptmMq290fw9BGkw JhQg== MIME-Version: 1.0 Received: by 10.60.25.104 with SMTP id b8mr3139126oeg.140.1349323891267; Wed, 03 Oct 2012 21:11:31 -0700 (PDT) Received: by 10.182.49.130 with HTTP; Wed, 3 Oct 2012 21:11:31 -0700 (PDT) In-Reply-To: References: Date: Thu, 4 Oct 2012 12:11:31 +0800 Message-ID: From: cz li To: freebsd-hackers Content-Type: text/plain; charset=UTF-8 Subject: Re: help me,my virtualbox run error. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 04:11:32 -0000 I follow the above said to did. FreeBSDHos# ls -ld /tmp drwxrwxrwt 20 root wheel 2560 Oct 4 11:53 /tmp I use ROOT user login.but, can not run. I installed on this machine GNOME.Very strange,I can only use the ROOT user log on normally.New user logs on, the user interface is not responding.I can only used ROOT user. Thank you 2012/10/3 Tom Evans : > On Wed, Oct 3, 2012 at 4:16 PM, cz li wrote: >> my os version is freebsd 9.0.I installed virtualbox 4.0.2.Installation >> no error.but, can not run.error infromation: >> "Failed to create the VirtualBox COM object. >> >> The application will now terminate. >> >> >> >> Callee RC: NS_ERROR_FACTORY_NOT_REGISTERED (0x80040154) >> " > > > https://www.virtualbox.org/ticket/2335 > > > Cheers > > Tom From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 4 05:52:30 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F178106564A for ; Thu, 4 Oct 2012 05:52:30 +0000 (UTC) (envelope-from tim@kientzle.com) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id C628D8FC4C for ; Thu, 4 Oct 2012 05:52:27 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id q945qK0s096847; Thu, 4 Oct 2012 05:52:20 GMT (envelope-from tim@kientzle.com) Received: from [192.168.2.143] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id fnwarrsb455mjzxxerue5m9ccn; Thu, 04 Oct 2012 05:52:20 +0000 (UTC) (envelope-from tim@kientzle.com) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=iso-8859-1 From: Tim Kientzle In-Reply-To: <506C866C.3050509@gentoo.org> Date: Wed, 3 Oct 2012 22:52:18 -0700 Content-Transfer-Encoding: 7bit Message-Id: <8241FA15-1F31-46EC-95D0-9AB9ECEB9832@kientzle.com> References: <5069C9FC.6020400@brandonfa.lk> <87549776-9051-4B4B-8D53-DAE6D51C2A94@kientzle.com> <20121002083634.3103fe958508a4026384ac96@yamagi.org> <506C866C.3050509@gentoo.org> To: Richard Yao X-Mailer: Apple Mail (2.1278) Cc: freebsd-hackers@freebsd.org Subject: Re: SMP Version of tar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 05:52:30 -0000 > Someone might want to ask if parallelizing tar is even possible. Answer: Yes. Here's a simple parallel version of tar: find . | cpio -o -H ustar | gzip > outfile.tgz There are definitely other approaches. Tim From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 4 16:11:23 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 71256106566C; Thu, 4 Oct 2012 16:11:23 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) by mx1.freebsd.org (Postfix) with ESMTP id 3B3B78FC08; Thu, 4 Oct 2012 16:11:22 +0000 (UTC) Received: from [209.249.190.124] (port=50363 helo=gnnmac.hudson-trading.com) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80) (envelope-from ) id 1TJo1C-0007o9-Br; Thu, 04 Oct 2012 12:11:22 -0400 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) From: George Neville-Neil In-Reply-To: <201210021037.27762.jhb@freebsd.org> Date: Thu, 4 Oct 2012 12:11:21 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <5BC1F38C-C354-4A01-B809-A8FE64824ABD@neville-neil.com> References: <201210020750.23358.jhb@freebsd.org> <201210021037.27762.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1498) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Mailman-Approved-At: Thu, 04 Oct 2012 16:20:45 +0000 Cc: Garrett Cooper , freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org, "Simon J. Gerraty" Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 16:11:23 -0000 On Oct 2, 2012, at 10:37 , John Baldwin wrote: > This is very non-obvious to the public at large (e.g. there was no = public=20 > response to one group's inquiry about the second ATF import for = example). =20 > Also, given that you had no idea that sgf@ and obrien@ were working on=20= > importing NetBSD's bmake as a prerequisite for ATF, it seems that = whatever=20 > discussions were held were not very detailed at best. I think it = would be=20 > good to have the various folks working on ATF to at least summarize = the=20 > current state of things and sketch out some sort of plan or roadmap = for future=20 > work in a public forum (such as atf@, though a summary mail would be = quite=20 > appropriate for arch@). I take partial responsibility for the privacy of the discussions = hitherto. My apologies, it should have be moved out onto a public list sooner. But, I would like to drive this to a solution on arch@. We don't have = an atf@, but we do have a test@ and testing@. We have too many mailing lists already, so let's finish this up here if we can and then=20 continue talking on testing@. Best, George From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 4 16:29:11 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C26A81065670; Thu, 4 Oct 2012 16:29:11 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (m209-73.dsl.rawbw.com [198.144.209.73]) by mx1.freebsd.org (Postfix) with ESMTP id 90E7A8FC16; Thu, 4 Oct 2012 16:29:11 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.5/8.14.5) with ESMTP id q94GTAfW038242; Thu, 4 Oct 2012 09:29:10 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.5/8.14.5/Submit) id q94GTAFw038241; Thu, 4 Oct 2012 09:29:10 -0700 (PDT) (envelope-from david) Date: Thu, 4 Oct 2012 09:29:10 -0700 From: David Wolfskill To: George Neville-Neil Message-ID: <20121004162910.GQ23688@albert.catwhisker.org> References: <201210020750.23358.jhb@freebsd.org> <201210021037.27762.jhb@freebsd.org> <5BC1F38C-C354-4A01-B809-A8FE64824ABD@neville-neil.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="sxhug0Teuf3tiWmo" Content-Disposition: inline In-Reply-To: <5BC1F38C-C354-4A01-B809-A8FE64824ABD@neville-neil.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org, "Simon J. Gerraty" , freebsd-arch@freebsd.org Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 16:29:11 -0000 --sxhug0Teuf3tiWmo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 04, 2012 at 12:11:21PM -0400, George Neville-Neil wrote: > ... > But, I would like to drive this to a solution on arch@. We don't have an > atf@, but we do have a test@ and testing@. We have too many mailing > lists already, so let's finish this up here if we can and then=20 > continue talking on testing@. > ... Before folks get too excited about test@ & testing@: * test@ is for testing ability to send mail to FreeBSD.org (mailing lists). * testing@ *used* to exist, but was retired for lack of relevant traffic. * There's little difference in effort in resurrecting testing@ vs. creating atf@. Peace, david (current hat: part of postmaster@freebsd.org) --=20 David H. Wolfskill david@catwhisker.org Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --sxhug0Teuf3tiWmo Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlBtuVUACgkQmprOCmdXAD20EQCdH48KJDsJRSbTis+lKMJaMhKV +h8AniqLHtULN4kBAWZT2/nwSFe0/a5b =nuve -----END PGP SIGNATURE----- --sxhug0Teuf3tiWmo-- From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 4 16:42:36 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 14F80106564A; Thu, 4 Oct 2012 16:42:36 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9CE138FC14; Thu, 4 Oct 2012 16:42:35 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so968738oag.13 for ; Thu, 04 Oct 2012 09:42:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=cfhiCnX0umtxbR+tLgCvwAzPiDUiLQOAN+xvx1YK3so=; b=b+IlJ8vAbGB0G4QnzLsHs3lY9RPhDZYXLOKwOR8aLZMVjLUFF6RTohTfJxokaSr3Az jt99x8gyAvKdJQagpmCo69WsEvQjHTeuEVyX6yACyZX0k221+7HZlJk1peqCnJVLAMAH X6v/uUQbvqGTkjT/BGEFa9y5aQF2xMqjsAps3fgld3tRkB4C5Y2B7gyEcQUFOxuktWvm OwHKRtSnYsUkGPXayLT/RpHO+nGdJdMKzRg5PAoB5CbNNJgdyGTw+ClPdG//vBs5h7t0 8uRf9XO2gcSCvka902sRiP7l/TRxqODhOocGnV0kXsbDa0li9q82QdccHtn3PP0YCbSV Nv2g== MIME-Version: 1.0 Received: by 10.182.52.3 with SMTP id p3mr4726552obo.56.1349368949294; Thu, 04 Oct 2012 09:42:29 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Thu, 4 Oct 2012 09:42:29 -0700 (PDT) In-Reply-To: <201210021037.27762.jhb@freebsd.org> References: <201210020750.23358.jhb@freebsd.org> <201210021037.27762.jhb@freebsd.org> Date: Thu, 4 Oct 2012 09:42:29 -0700 Message-ID: From: Garrett Cooper To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, "Simon J. Gerraty" , freebsd-arch@freebsd.org Subject: Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 16:42:36 -0000 On Tue, Oct 2, 2012 at 7:37 AM, John Baldwin wrote: > On Tuesday, October 02, 2012 10:29:49 am Garrett Cooper wrote: >> On Tue, Oct 2, 2012 at 4:50 AM, John Baldwin wrote: >> >> ... >> >> > This sounds like a superior approach. It doesn't break any current use >> > cases while giving the ability to build multiple programs in the few >> > places that need it. It sounds like there are a few places under gnu/ >> > from Garrett's reply that might be able to make use of this as well. >> >> For the record, gnu/cc/cc_tools/Makefile is where I first spotted a >> potential "bsd.progs.mk" candidate. Most of the other code doesn't >> care given how things are organized in our source tree. >> >> > BTW, one general comment. There seem to be two completely independent >> > groups of folks working on ATF (e.g. there have been two different >> > imports of ATF into the tree in two different locations IIRC, and now >> > we have two different sets of patches to our system makefiles). >> > >> > Are these two groups talking to each other at all? I know in May that >> > many folks (certainly multiple vendors) are interested in ATF, and it >> > seems that both Juniper and Isilon have ported ATF internally. It seems >> > that it might be good for the two groups to work together to avoid >> > stomping on each other's toes. It seems there are some differences in >> > the two approaches that merit working out to avoid a lot of wasted >> > effort on both sides. >> >> Both parties (Isilon/Juniper) are converging on the ATF porting work >> that Giorgos/myself have done after talking at the FreeBSD Foundation >> meet-n-greet. I have contributed all of the patches that I have other >> to marcel for feedback. > > This is very non-obvious to the public at large (e.g. there was no public > response to one group's inquiry about the second ATF import for example). > Also, given that you had no idea that sgf@ and obrien@ were working on > importing NetBSD's bmake as a prerequisite for ATF, it seems that whatever > discussions were held were not very detailed at best. I think it would be > good to have the various folks working on ATF to at least summarize the > current state of things and sketch out some sort of plan or roadmap for future > work in a public forum (such as atf@, though a summary mail would be quite > appropriate for arch@). I'm in part to blame for this. There was some discussion -- but not at length; unfortunately no one from Juniper was present at the meet and greet; the information I got was second hand; I didn't follow up to figure out the exact details / clarify what I had in mind with the appropriate parties. That being said, I *sort* of understand what's going on now, although I'm still guessing as I haven't received any FreeBSD internal (developers@, etc) emails and all the discussion I have so far is between gnn@, marcel@, gibbs@, mlaier@, and mdf@. For all intents and purposes I've been paused for a few weeks because of other things, so no harm, no foul, but I'd like to know what all is being contributed back from Juniper in terms of tests, ATF integration into the build system (which I've given back to marcel@/gnn@, but haven't received feedback for yet -- probably because they're busy), etc so I can better avoid duplicated effort and help the cause of creating a maintainable/testable FreeBSD. As far as what Isilon is contributing back, I wouldn't look any further than my perforce repo; the original goal as of last BSDCan was to contribute back `isi_check` (custom wrapper around GNU libcheck), and maybe some of our tests are written for isi_check: the problem with that plan is that it depends on GNU [lib]check -- yet another test infrastructure -- diverges us more unnecessarily from NetBSD (and there are several things we want to grab from NetBSD and contribute back instead of forging ahead down our own path), the tests would need to be audited and cleaned up to use generic macros and check for generic functionality, etc. Also, it would help to have generic system tests similar to LTP's breadth in the tree (so that way we can avoid breakage before things are committed to current). There are some functional gaps that I like to fill in with ATF that GNU libcheck does well [with fixtures] and there are some inconsistencies between the ATF C and C++ bindings I'm working on enhancing More details about what I planned on doing can be found here: http://wiki.freebsd.org/TestingFreeBSD -- although it deserves updating when looking at the structure and dealing with the "patches" (I need to update it to "just work" with the latest current). Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 4 16:44:31 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2DC671065673; Thu, 4 Oct 2012 16:44:31 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id AB1BE8FC1F; Thu, 4 Oct 2012 16:44:29 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so971775oag.13 for ; Thu, 04 Oct 2012 09:44:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=OIcPlwJWYOyaCDKC5Jp/d/+vu8n+MMMOb9pLTG+txqg=; b=Fg+8tf+ks2qTkyrHzMLDFjur6EFrOQE7PwJvfK5zV/lsnEmKSkgS7AFj/W6LQI0r4F ja/XyKBYTbUdEFsc/H+R4Bl2v/kio/RyCI9HD8xfWc/n2+oz+87VySXO9oyDH92qHrTa qpfKi/bN0jtF2CJPDEDN8A/n1/1Cdp+gwQBebiApSUClEzBrpSL7CnkFI6pCEgdKsNLX pog19tIkktDhwkmHdegg+YllU2AeNEdDpbZSLxfDo+2HCK8/XkftOdUKXC3ykqF4M4Aq qidjHDEEQKkRyiK3rrOEQk09E8++1O5b9aoZlrQlMfpMeYTwEBq/EkqInANL4Xozx/1t SyLA== MIME-Version: 1.0 Received: by 10.182.10.6 with SMTP id e6mr4831252obb.16.1349369069022; Thu, 04 Oct 2012 09:44:29 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Thu, 4 Oct 2012 09:44:28 -0700 (PDT) In-Reply-To: <20121004162910.GQ23688@albert.catwhisker.org> References: <201210020750.23358.jhb@freebsd.org> <201210021037.27762.jhb@freebsd.org> <5BC1F38C-C354-4A01-B809-A8FE64824ABD@neville-neil.com> <20121004162910.GQ23688@albert.catwhisker.org> Date: Thu, 4 Oct 2012 09:44:28 -0700 Message-ID: From: Garrett Cooper To: David Wolfskill Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, George Neville-Neil , freebsd-arch@freebsd.org, "Simon J. Gerraty" Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 16:44:31 -0000 On Thu, Oct 4, 2012 at 9:29 AM, David Wolfskill wrote: > On Thu, Oct 04, 2012 at 12:11:21PM -0400, George Neville-Neil wrote: >> ... >> But, I would like to drive this to a solution on arch@. We don't have an >> atf@, but we do have a test@ and testing@. We have too many mailing >> lists already, so let's finish this up here if we can and then >> continue talking on testing@. >> ... > > Before folks get too excited about test@ & testing@: > > * test@ is for testing ability to send mail to FreeBSD.org (mailing > lists). > > * testing@ *used* to exist, but was retired for lack of relevant > traffic. > > * There's little difference in effort in resurrecting testing@ vs. > creating atf@. I think that resurrecting testing@ makes more sense as creating an atf specific list seems a bit too focused on ATF, primarily because atf is being "partially superseded" by kyua (pronounced Q-A) in the near future. Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 4 17:20:58 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 979381065670 for ; Thu, 4 Oct 2012 17:20:58 +0000 (UTC) (envelope-from carl.r.delsey@intel.com) Received: from mga03.intel.com (mga03.intel.com [143.182.124.21]) by mx1.freebsd.org (Postfix) with ESMTP id 710E68FC0C for ; Thu, 4 Oct 2012 17:20:58 +0000 (UTC) Received: from azsmga002.ch.intel.com ([10.2.17.35]) by azsmga101.ch.intel.com with ESMTP; 04 Oct 2012 10:20:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.80,537,1344236400"; d="scan'208";a="152468880" Received: from crdelsey-fbsd.ch.intel.com (HELO [10.2.105.127]) ([10.2.105.127]) by AZSMGA002.ch.intel.com with ESMTP; 04 Oct 2012 10:20:52 -0700 Message-ID: <506DC574.9010300@intel.com> Date: Thu, 04 Oct 2012 10:20:52 -0700 From: Carl Delsey User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120724 Thunderbird/13.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: No bus_space_read_8 on x86 ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 17:20:58 -0000 I noticed that the bus_space_*_8 functions are unimplemented for x86. Looking at the code, it seems this is intentional. Is this done because on 32-bit systems we don't know, in the general case, whether to read the upper or lower 32-bits first? If that's the reason, I was thinking we could provide two implementations for i386: bus_space_read_8_upper_first and bus_space_read_8_lower_first. For amd64 we would just have bus_space_read_8 Anybody who wants to use bus_space_read_8 in their file would do something like: #define BUS_SPACE_8_BYTES LOWER_FIRST or #define BUS_SPACE_8_BYTES UPPER_FIRST whichever is appropriate for their hardware. This would go in their source file before including bus.h and we would take care of mapping to the correct implementation. With the prevalence of 64-bit registers these days, if we don't provide an implementation, I expect many drivers will end up rolling their own. If this seems like a good idea, I'll happily whip up a patch and submit it. From owner-freebsd-hackers@FreeBSD.ORG Thu Oct 4 17:24:06 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 54B15106564A; Thu, 4 Oct 2012 17:24:06 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from exprod7og119.obsmtp.com (exprod7og119.obsmtp.com [64.18.2.16]) by mx1.freebsd.org (Postfix) with ESMTP id ADE8E8FC0A; Thu, 4 Oct 2012 17:24:04 +0000 (UTC) Received: from P-EMHUB03-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob119.postini.com ([64.18.6.12]) with SMTP ID DSNKUG3GMxBnovHKF+pTeUy1ZIIkq/8e3SrP@postini.com; Thu, 04 Oct 2012 10:24:05 PDT Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB03-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Thu, 4 Oct 2012 10:22:05 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id q94HM4h44676; Thu, 4 Oct 2012 10:22:04 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 0518958093; Thu, 4 Oct 2012 10:22:04 -0700 (PDT) To: Garrett Cooper In-Reply-To: References: <201210020750.23358.jhb@freebsd.org> <201210021037.27762.jhb@freebsd.org> Comments: In-reply-to: Garrett Cooper message dated "Thu, 04 Oct 2012 09:42:29 -0700." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Thu, 4 Oct 2012 10:22:03 -0700 Message-ID: <20121004172204.0518958093@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-Mailman-Approved-At: Thu, 04 Oct 2012 18:23:30 +0000 Cc: freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 17:24:06 -0000 On Thu, 4 Oct 2012 09:42:29 -0700, Garrett Cooper writes: >I'd like to know what all is >being contributed back from Juniper in terms of tests, ATF integration >into the build system (which I've given back to marcel@/gnn@, but We aim to contribute build improvments, and integration of ATF was part of that. We had planned on doing the ATF import etc, but didn't want to re-invent wheels by trying to make it work without bmake. Or making a dog's breakfast out of bsd.*.mk Speaking of which, the initial commit (which should happen "real soon now" ;-) is a minimal set of changes to allow buildworld etc using bmake and allow folk who want to, to install bmake as make - as discussed at the last devsummit in Cambridge. Anyway, back to ATF, as mentioned earlier in this thread, I use atf.test.mk in our build rather than netbsd's bsd.test.mk, and we put all the test makefiles in a tests/ subdir of the lib or prog in question. This has important ramifications when it comes to wanting to build the tree in meta mode and have unit-tests build and run as an integral part of the build (or at least the option of doing that). As for meta mode, there is a projects/bmake branch which is a bit out of sync with head, but which I think has meta mode enabled (my internal mirror of it does ;-). It isn't ready for prime-time yet, a lot of the stuff in local.sys.mk there needs to migrate to sys.mk or similar, but that should probably wait until bmake is the default make, and there is also the need for more discussion here. But with a couple of env variables set, people should be able to play with it, to see what we are talking about. The next steps will focus on being able to have bmake the default - which means dealing with ports. I've a "patch", that's air-quotes, because I don't think a patch will suffice for a large moving target, rather its a script to run against a ports tree. Once the ports folk are happy that a bmake flavored ports tree can be built and used ok on an older base system, the final cutover can be planned. Hope that helps --sjg From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 5 00:31:38 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A5720106566B; Fri, 5 Oct 2012 00:31:38 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (lor.one-eyed-alien.net [69.66.77.232]) by mx1.freebsd.org (Postfix) with ESMTP id 56A748FC08; Fri, 5 Oct 2012 00:31:37 +0000 (UTC) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.5/8.14.5) with ESMTP id q950VaGE084431; Thu, 4 Oct 2012 19:31:36 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.5/8.14.5/Submit) id q950VaaX084430; Thu, 4 Oct 2012 19:31:36 -0500 (CDT) (envelope-from brooks) Date: Thu, 4 Oct 2012 19:31:36 -0500 From: Brooks Davis To: John Baldwin Message-ID: <20121005003136.GB84375@lor.one-eyed-alien.net> References: <20120924213137.GA76898@lor.one-eyed-alien.net> <201209250841.34134.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IiVenqGWf+H9Y6IX" Content-Disposition: inline In-Reply-To: <201209250841.34134.jhb@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org, hackers@freebsd.org Subject: Re: vendor import questions X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 00:31:38 -0000 --IiVenqGWf+H9Y6IX Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Sep 25, 2012 at 08:41:34AM -0400, John Baldwin wrote: > On Monday, September 24, 2012 5:31:37 pm Brooks Davis wrote: > > As part of switching to NetBSD's mtree I plan to import their versions > > of a few files that are part of libc (for example all the bits of > > vis/unvis). I would like to do that via a vendor import, but I'm unsure > > where to put the files and how to tag them. For mtree itself the right > > place is clearly base/vendor/NetBSD/mtree/dist, but we don't seem to > > have a good example for libc bits. > >=20 > > There is currently a base/vendor/NetBSD/dist directory containing a > > (very) partial source tree, but it seems to be unused in recent times. > > If I did import into that tree, the next question would be how to tag > > the import. The base/vendor/NetBSD/fparseln_19990920/ directory shows > > one seemingly sensible example, but I don't like the resulting explosion > > of top level directories. I also worry that having mixed versions in t= he > > libc directory would make any attempt at sensible merging difficult > > since we'd have to put mergeinfo on files. > >=20 > > An additional issue is where to put the files in the source tree. > > Precedent seems to favor direct copies to src/lib/libc/gen etc. In some > > ways I think the optimal solution would be to put the bits in contrib > > in feature specific directories like contrib/libc/vis, but that might > > be annoying for some consumers. That being said, the existence if > > src/include means you can't simply check out libc so it's probably ok to > > add more locations in the source tree for a good cause. > >=20 > > What's the right way to go here? >=20 > libc already has contrib bits (contrib/gdtoa). I think something like > contrib/NetBSD/libc/ might be fine. The problem I have with just > 'contrib/libc' is that it is ambiguous. OTOH, the contrib/NetBSD/libc > path isn't too pretty either. One option would be to merge directly from > the vendor area into src/lib/libc. One other option might be to just > do src/contrib/vis if it is only for 'vis' files. I'm leaning towards src/contrib/libc-vis. That would also work well in vendor/NetBSD since I could do vendor/NetBSD/libc-vis/dist. -- Brooks --IiVenqGWf+H9Y6IX Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iD8DBQFQbipnXY6L6fI4GtQRAk8sAKCgEdBSVKoPBY3X/cALH9Kac3IVLQCgqPrC 5Ib9BrqkByqsIk9ph77vEo0= =nzZf -----END PGP SIGNATURE----- --IiVenqGWf+H9Y6IX-- From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 5 00:31:38 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A5720106566B; Fri, 5 Oct 2012 00:31:38 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (lor.one-eyed-alien.net [69.66.77.232]) by mx1.freebsd.org (Postfix) with ESMTP id 56A748FC08; Fri, 5 Oct 2012 00:31:37 +0000 (UTC) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.5/8.14.5) with ESMTP id q950VaGE084431; Thu, 4 Oct 2012 19:31:36 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.5/8.14.5/Submit) id q950VaaX084430; Thu, 4 Oct 2012 19:31:36 -0500 (CDT) (envelope-from brooks) Date: Thu, 4 Oct 2012 19:31:36 -0500 From: Brooks Davis To: John Baldwin Message-ID: <20121005003136.GB84375@lor.one-eyed-alien.net> References: <20120924213137.GA76898@lor.one-eyed-alien.net> <201209250841.34134.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IiVenqGWf+H9Y6IX" Content-Disposition: inline In-Reply-To: <201209250841.34134.jhb@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org, hackers@freebsd.org Subject: Re: vendor import questions X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 00:31:38 -0000 --IiVenqGWf+H9Y6IX Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Sep 25, 2012 at 08:41:34AM -0400, John Baldwin wrote: > On Monday, September 24, 2012 5:31:37 pm Brooks Davis wrote: > > As part of switching to NetBSD's mtree I plan to import their versions > > of a few files that are part of libc (for example all the bits of > > vis/unvis). I would like to do that via a vendor import, but I'm unsure > > where to put the files and how to tag them. For mtree itself the right > > place is clearly base/vendor/NetBSD/mtree/dist, but we don't seem to > > have a good example for libc bits. > >=20 > > There is currently a base/vendor/NetBSD/dist directory containing a > > (very) partial source tree, but it seems to be unused in recent times. > > If I did import into that tree, the next question would be how to tag > > the import. The base/vendor/NetBSD/fparseln_19990920/ directory shows > > one seemingly sensible example, but I don't like the resulting explosion > > of top level directories. I also worry that having mixed versions in t= he > > libc directory would make any attempt at sensible merging difficult > > since we'd have to put mergeinfo on files. > >=20 > > An additional issue is where to put the files in the source tree. > > Precedent seems to favor direct copies to src/lib/libc/gen etc. In some > > ways I think the optimal solution would be to put the bits in contrib > > in feature specific directories like contrib/libc/vis, but that might > > be annoying for some consumers. That being said, the existence if > > src/include means you can't simply check out libc so it's probably ok to > > add more locations in the source tree for a good cause. > >=20 > > What's the right way to go here? >=20 > libc already has contrib bits (contrib/gdtoa). I think something like > contrib/NetBSD/libc/ might be fine. The problem I have with just > 'contrib/libc' is that it is ambiguous. OTOH, the contrib/NetBSD/libc > path isn't too pretty either. One option would be to merge directly from > the vendor area into src/lib/libc. One other option might be to just > do src/contrib/vis if it is only for 'vis' files. I'm leaning towards src/contrib/libc-vis. That would also work well in vendor/NetBSD since I could do vendor/NetBSD/libc-vis/dist. -- Brooks --IiVenqGWf+H9Y6IX Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iD8DBQFQbipnXY6L6fI4GtQRAk8sAKCgEdBSVKoPBY3X/cALH9Kac3IVLQCgqPrC 5Ib9BrqkByqsIk9ph77vEo0= =nzZf -----END PGP SIGNATURE----- --IiVenqGWf+H9Y6IX-- From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 5 16:17:37 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 49B701065672; Fri, 5 Oct 2012 16:17:37 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 1B3A08FC08; Fri, 5 Oct 2012 16:17:37 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 72BE7B94B; Fri, 5 Oct 2012 12:17:36 -0400 (EDT) From: John Baldwin To: Brooks Davis Date: Fri, 5 Oct 2012 11:47:47 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p20; KDE/4.5.5; amd64; ; ) References: <20120924213137.GA76898@lor.one-eyed-alien.net> <201209250841.34134.jhb@freebsd.org> <20121005003136.GB84375@lor.one-eyed-alien.net> In-Reply-To: <20121005003136.GB84375@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201210051147.47411.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 05 Oct 2012 12:17:36 -0400 (EDT) Cc: freebsd-hackers@freebsd.org, hackers@freebsd.org Subject: Re: vendor import questions X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 16:17:37 -0000 On Thursday, October 04, 2012 8:31:36 pm Brooks Davis wrote: > On Tue, Sep 25, 2012 at 08:41:34AM -0400, John Baldwin wrote: > > On Monday, September 24, 2012 5:31:37 pm Brooks Davis wrote: > > > As part of switching to NetBSD's mtree I plan to import their versions > > > of a few files that are part of libc (for example all the bits of > > > vis/unvis). I would like to do that via a vendor import, but I'm unsure > > > where to put the files and how to tag them. For mtree itself the right > > > place is clearly base/vendor/NetBSD/mtree/dist, but we don't seem to > > > have a good example for libc bits. > > > > > > There is currently a base/vendor/NetBSD/dist directory containing a > > > (very) partial source tree, but it seems to be unused in recent times. > > > If I did import into that tree, the next question would be how to tag > > > the import. The base/vendor/NetBSD/fparseln_19990920/ directory shows > > > one seemingly sensible example, but I don't like the resulting explosion > > > of top level directories. I also worry that having mixed versions in the > > > libc directory would make any attempt at sensible merging difficult > > > since we'd have to put mergeinfo on files. > > > > > > An additional issue is where to put the files in the source tree. > > > Precedent seems to favor direct copies to src/lib/libc/gen etc. In some > > > ways I think the optimal solution would be to put the bits in contrib > > > in feature specific directories like contrib/libc/vis, but that might > > > be annoying for some consumers. That being said, the existence if > > > src/include means you can't simply check out libc so it's probably ok to > > > add more locations in the source tree for a good cause. > > > > > > What's the right way to go here? > > > > libc already has contrib bits (contrib/gdtoa). I think something like > > contrib/NetBSD/libc/ might be fine. The problem I have with just > > 'contrib/libc' is that it is ambiguous. OTOH, the contrib/NetBSD/libc > > path isn't too pretty either. One option would be to merge directly from > > the vendor area into src/lib/libc. One other option might be to just > > do src/contrib/vis if it is only for 'vis' files. > > I'm leaning towards src/contrib/libc-vis. That would also work well in > vendor/NetBSD since I could do vendor/NetBSD/libc-vis/dist. I think that is fine. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 5 16:17:37 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 49B701065672; Fri, 5 Oct 2012 16:17:37 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 1B3A08FC08; Fri, 5 Oct 2012 16:17:37 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 72BE7B94B; Fri, 5 Oct 2012 12:17:36 -0400 (EDT) From: John Baldwin To: Brooks Davis Date: Fri, 5 Oct 2012 11:47:47 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p20; KDE/4.5.5; amd64; ; ) References: <20120924213137.GA76898@lor.one-eyed-alien.net> <201209250841.34134.jhb@freebsd.org> <20121005003136.GB84375@lor.one-eyed-alien.net> In-Reply-To: <20121005003136.GB84375@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201210051147.47411.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 05 Oct 2012 12:17:36 -0400 (EDT) Cc: freebsd-hackers@freebsd.org, hackers@freebsd.org Subject: Re: vendor import questions X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 16:17:37 -0000 On Thursday, October 04, 2012 8:31:36 pm Brooks Davis wrote: > On Tue, Sep 25, 2012 at 08:41:34AM -0400, John Baldwin wrote: > > On Monday, September 24, 2012 5:31:37 pm Brooks Davis wrote: > > > As part of switching to NetBSD's mtree I plan to import their versions > > > of a few files that are part of libc (for example all the bits of > > > vis/unvis). I would like to do that via a vendor import, but I'm unsure > > > where to put the files and how to tag them. For mtree itself the right > > > place is clearly base/vendor/NetBSD/mtree/dist, but we don't seem to > > > have a good example for libc bits. > > > > > > There is currently a base/vendor/NetBSD/dist directory containing a > > > (very) partial source tree, but it seems to be unused in recent times. > > > If I did import into that tree, the next question would be how to tag > > > the import. The base/vendor/NetBSD/fparseln_19990920/ directory shows > > > one seemingly sensible example, but I don't like the resulting explosion > > > of top level directories. I also worry that having mixed versions in the > > > libc directory would make any attempt at sensible merging difficult > > > since we'd have to put mergeinfo on files. > > > > > > An additional issue is where to put the files in the source tree. > > > Precedent seems to favor direct copies to src/lib/libc/gen etc. In some > > > ways I think the optimal solution would be to put the bits in contrib > > > in feature specific directories like contrib/libc/vis, but that might > > > be annoying for some consumers. That being said, the existence if > > > src/include means you can't simply check out libc so it's probably ok to > > > add more locations in the source tree for a good cause. > > > > > > What's the right way to go here? > > > > libc already has contrib bits (contrib/gdtoa). I think something like > > contrib/NetBSD/libc/ might be fine. The problem I have with just > > 'contrib/libc' is that it is ambiguous. OTOH, the contrib/NetBSD/libc > > path isn't too pretty either. One option would be to merge directly from > > the vendor area into src/lib/libc. One other option might be to just > > do src/contrib/vis if it is only for 'vis' files. > > I'm leaning towards src/contrib/libc-vis. That would also work well in > vendor/NetBSD since I could do vendor/NetBSD/libc-vis/dist. I think that is fine. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 5 16:17:41 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 18F531065673 for ; Fri, 5 Oct 2012 16:17:41 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id E45928FC14 for ; Fri, 5 Oct 2012 16:17:40 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 507A0B9A4; Fri, 5 Oct 2012 12:17:40 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Fri, 5 Oct 2012 12:08:45 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p20; KDE/4.5.5; amd64; ; ) References: <506DC574.9010300@intel.com> In-Reply-To: <506DC574.9010300@intel.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201210051208.45550.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 05 Oct 2012 12:17:40 -0400 (EDT) Cc: Carl Delsey Subject: Re: No bus_space_read_8 on x86 ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 16:17:41 -0000 On Thursday, October 04, 2012 1:20:52 pm Carl Delsey wrote: > I noticed that the bus_space_*_8 functions are unimplemented for x86. > Looking at the code, it seems this is intentional. > > Is this done because on 32-bit systems we don't know, in the general > case, whether to read the upper or lower 32-bits first? > > If that's the reason, I was thinking we could provide two > implementations for i386: bus_space_read_8_upper_first and > bus_space_read_8_lower_first. For amd64 we would just have bus_space_read_8 > > Anybody who wants to use bus_space_read_8 in their file would do > something like: > #define BUS_SPACE_8_BYTES LOWER_FIRST > or > #define BUS_SPACE_8_BYTES UPPER_FIRST > whichever is appropriate for their hardware. > > This would go in their source file before including bus.h and we would > take care of mapping to the correct implementation. > > With the prevalence of 64-bit registers these days, if we don't provide > an implementation, I expect many drivers will end up rolling their own. > > If this seems like a good idea, I'll happily whip up a patch and submit it. I think cxgb* already have an implementation. For amd64 we should certainly have bus_space_*_8(), at least for SYS_RES_MEMORY. I think they should fail for SYS_RES_IOPORT. I don't think we can force a compile-time error though, would just have to return -1 on reads or some such? -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 6 11:20:18 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 21C7B106566B; Sat, 6 Oct 2012 11:20:18 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1AFFC8FC0C; Sat, 6 Oct 2012 11:20:16 +0000 (UTC) Received: by mail-we0-f182.google.com with SMTP id x43so1983541wey.13 for ; Sat, 06 Oct 2012 04:20:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=G5XdW5gh/GjbNrWuIQkDnuWwceaifLYUubBEB8ftJrE=; b=zVmv4edKO66V/djE0bNJuGYHbUZ6Yg0Bme3OjaQbUjSFXl7sUr2ui7ON97BaigI5aV ztbr+6uTI4ZzTjPaAqRyLVDxfXc1a33Igma6roIJ/RMmIvOkeLojfq+h+FSZlqelzwUc uJS8QLy4md7/Tr/QbWdQIJO0nnLK5+2RqOJFoaJVFUW7mprFuTKu2Cj1ILdhIMAcuyTI 5M11c6YuSOwttqHP1L2xOeHrGa8/BMs7Udfz0sRaqfrvzpt+4hKJdgm+k8t8HprHbpAD 4U/ztalBhrWjkLmax3VU4UWcqlnBX8nBp+ndA0BJOC8WWRuTSLdQN2auIA34BT6XOrdm RKGQ== Received: by 10.216.207.163 with SMTP id n35mr6807055weo.220.1349522415533; Sat, 06 Oct 2012 04:20:15 -0700 (PDT) Received: from [10.0.0.86] ([93.152.184.10]) by mx.google.com with ESMTPS id cl8sm7638401wib.10.2012.10.06.04.20.13 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 06 Oct 2012 04:20:14 -0700 (PDT) Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) Content-Type: text/plain; charset=windows-1252 From: Nikolay Denev In-Reply-To: <1666343702.1682678.1349300219198.JavaMail.root@erie.cs.uoguelph.ca> Date: Sat, 6 Oct 2012 14:20:11 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: <3E7BCFB4-6EE6-48F5-ACA7-A615F3CE5BAC@gmail.com> References: <1666343702.1682678.1349300219198.JavaMail.root@erie.cs.uoguelph.ca> To: Rick Macklem X-Mailer: Apple Mail (2.1498) X-Mailman-Approved-At: Sat, 06 Oct 2012 13:36:21 +0000 Cc: freebsd-fs@freebsd.org, rmacklem@freebsd.org, hackers@freebsd.org, Garrett Wollman Subject: Re: NFS server bottlenecks X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Oct 2012 11:20:18 -0000 On Oct 4, 2012, at 12:36 AM, Rick Macklem wrote: > Garrett Wollman wrote: >> <> said: >>=20 >>>> Simple: just use a sepatate mutex for each list that a cache entry >>>> is on, rather than a global lock for everything. This would reduce >>>> the mutex contention, but I'm not sure how significantly since I >>>> don't have the means to measure it yet. >>>>=20 >>> Well, since the cache trimming is removing entries from the lists, I >>> don't >>> see how that can be done with a global lock for list updates? >>=20 >> Well, the global lock is what we have now, but the cache trimming >> process only looks at one list at a time, so not locking the list = that >> isn't being iterated over probably wouldn't hurt, unless there's some >> mechanism (that I didn't see) for entries to move from one list to >> another. Note that I'm considering each hash bucket a separate >> "list". (One issue to worry about in that case would be cache-line >> contention in the array of hash buckets; perhaps NFSRVCACHE_HASHSIZE >> ought to be increased to reduce that.) >>=20 > Yea, a separate mutex for each hash list might help. There is also the > LRU list that all entries end up on, that gets used by the trimming = code. > (I think? I wrote this stuff about 8 years ago, so I haven't looked at > it in a while.) >=20 > Also, increasing the hash table size is probably a good idea, = especially > if you reduce how aggressively the cache is trimmed. >=20 >>> Only doing it once/sec would result in a very large cache when >>> bursts of >>> traffic arrives. >>=20 >> My servers have 96 GB of memory so that's not a big deal for me. >>=20 > This code was originally "production tested" on a server with 1Gbyte, > so times have changed a bit;-) >=20 >>> I'm not sure I see why doing it as a separate thread will improve >>> things. >>> There are N nfsd threads already (N can be bumped up to 256 if you >>> wish) >>> and having a bunch more "cache trimming threads" would just increase >>> contention, wouldn't it? >>=20 >> Only one cache-trimming thread. The cache trim holds the (global) >> mutex for much longer than any individual nfsd service thread has any >> need to, and having N threads doing that in parallel is why it's so >> heavily contended. If there's only one thread doing the trim, then >> the nfsd service threads aren't spending time either contending on = the >> mutex (it will be held less frequently and for shorter periods). >>=20 > I think the little drc2.patch which will keep the nfsd threads from > acquiring the mutex and doing the trimming most of the time, might be > sufficient. I still don't see why a separate trimming thread will be > an advantage. I'd also be worried that the one cache trimming thread > won't get the job done soon enough. >=20 > When I did production testing on a 1Gbyte server that saw a peak > load of about 100RPCs/sec, it was necessary to trim aggressively. > (Although I'd be tempted to say that a server with 1Gbyte is no > longer relevant, I recently recall someone trying to run FreeBSD > on a i486, although I doubt they wanted to run the nfsd on it.) >=20 >>> The only negative effect I can think of w.r.t. having the nfsd >>> threads doing it would be a (I believe negligible) increase in RPC >>> response times (the time the nfsd thread spends trimming the cache). >>> As noted, I think this time would be negligible compared to disk I/O >>> and network transit times in the total RPC response time? >>=20 >> With adaptive mutexes, many CPUs, lots of in-memory cache, and 10G >> network connectivity, spinning on a contended mutex takes a >> significant amount of CPU time. (For the current design of the NFS >> server, it may actually be a win to turn off adaptive mutexes -- I >> should give that a try once I'm able to do more testing.) >>=20 > Have fun with it. Let me know when you have what you think is a good = patch. >=20 > rick >=20 >> -GAWollman >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to >> "freebsd-hackers-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" I was doing some NFS testing with RELENG_9 machine and a Linux RHEL machine over 10G network, and noticed the same nfsd threads = issue. Previously I would read a 32G file locally on the FreeBSD ZFS/NFS server = with "dd if=3D/tank/32G.bin of=3D/dev/null bs=3D1M" to cache it = completely in ARC (machine has 196G RAM), then if I do this again locally I would get close to 4GB/sec read - = completely from the cache... But If I try to read the file over NFS from the Linux machine I would = only get about 100MB/sec speed, sometimes a bit more, and all of the nfsd threads are clearly visible in top. pmcstat also = showed the same mutex contention as in the original post. I've now applied the drc2 patch, and reruning the same test yields about = 960MB/s transfer over NFS=85 quite an improvement! From owner-freebsd-hackers@FreeBSD.ORG Sat Oct 6 22:32:58 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 62D9E106566B; Sat, 6 Oct 2012 22:32:58 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id A00018FC14; Sat, 6 Oct 2012 22:32:57 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap8EAA+xcFCDaFvO/2dsb2JhbABFhhG6GoIgAQEBBAEBASArIAsbDgoCAg0ZAikBCSYGCAIFBAEcAQOHZAumepF5gSGKLhqEZIESA5M+gi2BFY8ZgwmBRzQ X-IronPort-AV: E=Sophos;i="4.80,545,1344225600"; d="scan'208";a="182305646" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 06 Oct 2012 18:32:56 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 72EEAB4037; Sat, 6 Oct 2012 18:32:56 -0400 (EDT) Date: Sat, 6 Oct 2012 18:32:56 -0400 (EDT) From: Rick Macklem To: Nikolay Denev Message-ID: <895825217.1831774.1349562776418.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <3E7BCFB4-6EE6-48F5-ACA7-A615F3CE5BAC@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org, rmacklem@freebsd.org, hackers@freebsd.org, Garrett Wollman Subject: Re: NFS server bottlenecks X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Oct 2012 22:32:58 -0000 Nikolay Deney wrote: > On Oct 4, 2012, at 12:36 AM, Rick Macklem > wrote: >=20 > > Garrett Wollman wrote: > >> < >> said: > >> > >>>> Simple: just use a sepatate mutex for each list that a cache > >>>> entry > >>>> is on, rather than a global lock for everything. This would > >>>> reduce > >>>> the mutex contention, but I'm not sure how significantly since I > >>>> don't have the means to measure it yet. > >>>> > >>> Well, since the cache trimming is removing entries from the lists, > >>> I > >>> don't > >>> see how that can be done with a global lock for list updates? > >> > >> Well, the global lock is what we have now, but the cache trimming > >> process only looks at one list at a time, so not locking the list > >> that > >> isn't being iterated over probably wouldn't hurt, unless there's > >> some > >> mechanism (that I didn't see) for entries to move from one list to > >> another. Note that I'm considering each hash bucket a separate > >> "list". (One issue to worry about in that case would be cache-line > >> contention in the array of hash buckets; perhaps > >> NFSRVCACHE_HASHSIZE > >> ought to be increased to reduce that.) > >> > > Yea, a separate mutex for each hash list might help. There is also > > the > > LRU list that all entries end up on, that gets used by the trimming > > code. > > (I think? I wrote this stuff about 8 years ago, so I haven't looked > > at > > it in a while.) > > > > Also, increasing the hash table size is probably a good idea, > > especially > > if you reduce how aggressively the cache is trimmed. > > > >>> Only doing it once/sec would result in a very large cache when > >>> bursts of > >>> traffic arrives. > >> > >> My servers have 96 GB of memory so that's not a big deal for me. > >> > > This code was originally "production tested" on a server with > > 1Gbyte, > > so times have changed a bit;-) > > > >>> I'm not sure I see why doing it as a separate thread will improve > >>> things. > >>> There are N nfsd threads already (N can be bumped up to 256 if you > >>> wish) > >>> and having a bunch more "cache trimming threads" would just > >>> increase > >>> contention, wouldn't it? > >> > >> Only one cache-trimming thread. The cache trim holds the (global) > >> mutex for much longer than any individual nfsd service thread has > >> any > >> need to, and having N threads doing that in parallel is why it's so > >> heavily contended. If there's only one thread doing the trim, then > >> the nfsd service threads aren't spending time either contending on > >> the > >> mutex (it will be held less frequently and for shorter periods). > >> > > I think the little drc2.patch which will keep the nfsd threads from > > acquiring the mutex and doing the trimming most of the time, might > > be > > sufficient. I still don't see why a separate trimming thread will be > > an advantage. I'd also be worried that the one cache trimming thread > > won't get the job done soon enough. > > > > When I did production testing on a 1Gbyte server that saw a peak > > load of about 100RPCs/sec, it was necessary to trim aggressively. > > (Although I'd be tempted to say that a server with 1Gbyte is no > > longer relevant, I recently recall someone trying to run FreeBSD > > on a i486, although I doubt they wanted to run the nfsd on it.) > > > >>> The only negative effect I can think of w.r.t. having the nfsd > >>> threads doing it would be a (I believe negligible) increase in RPC > >>> response times (the time the nfsd thread spends trimming the > >>> cache). > >>> As noted, I think this time would be negligible compared to disk > >>> I/O > >>> and network transit times in the total RPC response time? > >> > >> With adaptive mutexes, many CPUs, lots of in-memory cache, and 10G > >> network connectivity, spinning on a contended mutex takes a > >> significant amount of CPU time. (For the current design of the NFS > >> server, it may actually be a win to turn off adaptive mutexes -- I > >> should give that a try once I'm able to do more testing.) > >> > > Have fun with it. Let me know when you have what you think is a good > > patch. > > > > rick > > > >> -GAWollman > >> _______________________________________________ > >> freebsd-hackers@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > >> To unsubscribe, send any mail to > >> "freebsd-hackers-unsubscribe@freebsd.org" > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to > > "freebsd-fs-unsubscribe@freebsd.org" >=20 > I was doing some NFS testing with RELENG_9 machine and > a Linux RHEL machine over 10G network, and noticed the same nfsd > threads issue. >=20 > Previously I would read a 32G file locally on the FreeBSD ZFS/NFS > server with "dd if=3D/tank/32G.bin of=3D/dev/null bs=3D1M" to cache it > completely in ARC (machine has 196G RAM), > then if I do this again locally I would get close to 4GB/sec read - > completely from the cache... >=20 > But If I try to read the file over NFS from the Linux machine I would > only get about 100MB/sec speed, sometimes a bit more, > and all of the nfsd threads are clearly visible in top. pmcstat also > showed the same mutex contention as in the original post. >=20 > I've now applied the drc2 patch, and reruning the same test yields > about 960MB/s transfer over NFS=E2=80=A6 quite an improvement! >=20 Sounds good. Hopefully Garrett can test it too and then it sounds like in can be committed. Someday I'll look at using separate mutexes for each of the hash buckets, which should reduce contention for the mutex for TCP. For UDP, there is one LRU list that all entries are on, so UDP is probably stuck using one mutex for now. Since this would be a more involved and risky patch, I think committing drc2.patch first and then doing this later, would make sense. Thanks for testing it, rick >=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org"