From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 2 00:36:51 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5AA8E1065673 for ; Sun, 2 Nov 2008 00:36:51 +0000 (UTC) (envelope-from freebsd.dev@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.157]) by mx1.freebsd.org (Postfix) with ESMTP id DF0CA8FC14 for ; Sun, 2 Nov 2008 00:36:50 +0000 (UTC) (envelope-from freebsd.dev@gmail.com) Received: by fg-out-1718.google.com with SMTP id l26so1526554fgb.35 for ; Sat, 01 Nov 2008 17:36:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type:content-transfer-encoding :content-disposition; bh=7XfYhQlpfsMtQBt8m9R9t97Q7Pz1meSul1vBvdC8k38=; b=mIe3g7MAkmkJeQN40xPqhTUuFHBBg51CKwgutttpgvoV5QvPHrXXlWo0i6HudfbWxP XuQD967XE5/Imu1zQ8JNjflW9IbcO760JYzEXJdpbJ0IRU1EXIhpzrZs4XNk1325CPdH si8hrrnvx6Th5jutUJjct5goGWnALO+LefbtE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type :content-transfer-encoding:content-disposition; b=Ou3cpBD5DZ2Ry1+f64eF1ytvE51288ph7C2wH5ZB5AuDL065rVm91E0957IMKUCKId 3HPWeutk0tdfa9Tw5FlYy5bZR3CWuxgN2GMj5ygswCFO0vlDndqngWid4SjSXDbLR8zc D5qvsmXqT+6pElErBYYjIhxrOtsGs+KbBFJyQ= Received: by 10.86.68.2 with SMTP id q2mr4684705fga.3.1225584523261; Sat, 01 Nov 2008 17:08:43 -0700 (PDT) Received: by 10.86.68.14 with HTTP; Sat, 1 Nov 2008 17:08:43 -0700 (PDT) Message-ID: <50cd4e5f0811011708t1f4fc665u592602d5085a6d56@mail.gmail.com> Date: Sat, 1 Nov 2008 19:08:43 -0500 From: "Biks N" To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: Need to optimize cutstom kernel hacks X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Nov 2008 00:36:51 -0000 Hi, To get started with FreeBSD kernel, I have been working on IP Packet compression. After numerous crashes and failures now everything looks good and stable. I am using kernel zlib routines to compress payload. However I think my implementation is not efficient at all. Here are the steps I am doing for compression: 1. copy Payload to empty buffer using m_copydata() function 2. call deflateInit2 () for deflate initialization 3. call deflate() for actual compression 4. copy the compressed data in buffer back to Payload I have to go through all above 4 steps for each packet! I think it will be lot faster and efficient if: * Somehow get away with deflateinit2() each time for each packet. * I can get to Payload pointer without using m_copydata() so that I don't need to copy data back and forth. Looking for your valuable suggestions and tips :) Bikrant From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 2 00:46:58 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 98DBE1065690 for ; Sun, 2 Nov 2008 00:46:58 +0000 (UTC) (envelope-from keramida@freebsd.org) Received: from igloo.linux.gr (igloo.linux.gr [62.1.205.36]) by mx1.freebsd.org (Postfix) with ESMTP id C4DE68FC17 for ; Sun, 2 Nov 2008 00:46:57 +0000 (UTC) (envelope-from keramida@freebsd.org) Received: from kobe.laptop (adsl158-209.kln.forthnet.gr [62.1.61.209]) (authenticated bits=128) by igloo.linux.gr (8.14.3/8.14.3/Debian-5) with ESMTP id mA20S3Jf017074 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 2 Nov 2008 02:28:09 +0200 Received: from kobe.laptop (kobe.laptop [127.0.0.1]) by kobe.laptop (8.14.3/8.14.3) with ESMTP id mA20S2uP006149; Sun, 2 Nov 2008 02:28:02 +0200 (EET) (envelope-from keramida@freebsd.org) Received: (from keramida@localhost) by kobe.laptop (8.14.3/8.14.3/Submit) id mA20S0LY006148; Sun, 2 Nov 2008 02:28:00 +0200 (EET) (envelope-from keramida@freebsd.org) From: Giorgos Keramidas To: Max Laier References: <200810071314.45922.max@love2party.net> <200811012114.43137.max@love2party.net> Date: Sun, 02 Nov 2008 02:27:59 +0200 Message-ID: <87zlkjvunk.fsf@kobe.laptop> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-MailScanner-ID: mA20S3Jf017074 X-Hellug-MailScanner: Found to be clean X-Hellug-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-4.501, required 5, autolearn=not spam, ALL_TRUSTED -1.80, AWL -0.10, BAYES_00 -2.60) X-Hellug-MailScanner-From: keramida@freebsd.org X-Spam-Status: No Cc: freebsd-hackers@freebsd.org, koitsu@freebsd.org, pjd@freebsd.org, Pete French , lhmwzy@gmail.com Subject: Re: du -A / -B options [Re: zfs quota question] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Nov 2008 00:46:58 -0000 On Sat, 1 Nov 2008 21:14:42 +0100, Max Laier wrote: > Hi, > a thread on freebsd-stable@ [1] about problems with du(1) and > compressed zfs filesystems got me looking for a possible solution. > Attached is a diff for du(1) that adds two new options: > > -A to display the apparent size of the file instead of the used blocks. > -B bsize to specify a custom blocksize. In particular one <512byte That's nice :) % Index: du.1 % =================================================================== % --- du.1 (revision 184513) % +++ du.1 (working copy) % @@ -60,6 +61,14 @@ % .Pp % The options are as follows: % .Bl -tag -width indent % +.It Fl A % +Display the apparent size instead of the diskusage. 'disk usage' should be two words here. % +This can be helpful to find sparse files and when operating on % +compressed volumes. ``This can be helpful when operating on sparse files or compressed volumes'' maybe? The fact that a difference between plain `du' and `du -A' output may be a nice hack to find sparse files is a cool side-effect, but that's not really the main driving force behind the change, right? :) % Index: du.c % =================================================================== % --- du.c (revision 184513) % +++ du.c (working copy) % @@ -86,27 +86,39 @@ % FTS *fts; % FTSENT *p; % off_t savednumber = 0; % - long blocksize; % + long blocksize = 0; savednumber is one of the existing style bugs. We shouldn't copy it, but initialize `blocksize' further down, near `depth = INT_MAX' and the rest of the initializations. > Thoughts? FWIW, I like the new options :) From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 2 10:36:51 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6C7F1065678 for ; Sun, 2 Nov 2008 10:36:51 +0000 (UTC) (envelope-from khomatapac@gmail.com) Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.227]) by mx1.freebsd.org (Postfix) with ESMTP id AC22F8FC31 for ; Sun, 2 Nov 2008 10:36:51 +0000 (UTC) (envelope-from khomatapac@gmail.com) Received: by rv-out-0506.google.com with SMTP id b25so1903789rvf.43 for ; Sun, 02 Nov 2008 02:36:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type; bh=VkgrVpHN4LN1ARMFx6iN1DrCyaaD49IurilSsl2RCfM=; b=Kcs1AFwMZnu90bop1B9CHws79V6qF+9sPCh2yEpbG/wh6Oj8AjBPQl1seMGCJknSPJ 7sywlrh9zO/EHNvJRQKCqVUu08CWK9W0Lz+35JtuC7VNFrRw8DXaY/FNTdIPLl7XWcPX Kz9jowCSsp1epyY+BuWge1dR3iW7yEPr6VocE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type; b=AVkCOZDcZ+LBaJ1vIYsJsZcSZaTdV7EGK/2BdXiFT2hJejlQzQvw9cqaV2E7XhtaLk 9fHMvo8MCTZ0a7tbGyGpk5DljI4zKTTa/c6KEQKkB3cIy/PrjX3tBQOseR3wzaQqNXob 8YtYFDjnRv4syGErV8eoCtInF5n+9x6mT6lVg= Received: by 10.142.128.6 with SMTP id a6mr6516215wfd.10.1225620679997; Sun, 02 Nov 2008 02:11:19 -0800 (PST) Received: by 10.142.191.12 with HTTP; Sun, 2 Nov 2008 02:11:19 -0800 (PST) Message-ID: <4775ce300811020211v7c3a31e0p20d421a97c3e0fcf@mail.gmail.com> Date: Sun, 2 Nov 2008 12:11:19 +0200 From: "Tapac Khoma" To: freebsd-hackers@freebsd.org MIME-Version: 1.0 X-Mailman-Approved-At: Sun, 02 Nov 2008 12:10:48 +0000 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: FreeBSD development X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Nov 2008 10:36:51 -0000 Hello! I use FreeBSD 6.1 as my desktop system. I like it very much. But there are some features I want my system have. I began to investigate of kernel structure and tried to write some devices drivers. So I want to know can I help for you in development of my favor system. And where I can find useful information about writing device drivers under FreeBSD. I will be wait for your answer. Thank you for attention. Taras Khoma From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 2 12:23:27 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78ECF1065686 for ; Sun, 2 Nov 2008 12:23:27 +0000 (UTC) (envelope-from rpaulo@gmail.com) Received: from ey-out-2122.google.com (ey-out-2122.google.com [74.125.78.24]) by mx1.freebsd.org (Postfix) with ESMTP id 06BC88FC16 for ; Sun, 2 Nov 2008 12:23:26 +0000 (UTC) (envelope-from rpaulo@gmail.com) Received: by ey-out-2122.google.com with SMTP id 6so652698eyi.7 for ; Sun, 02 Nov 2008 04:23:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:cc:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-mailer:sender; bh=6zkGbgRt3qvK9tcEPWoZS2x8ivwtMgeIBC9owlGsPrE=; b=KzcbRqks2rO8XpkORy5mfdD6HvJg8t9RrIDH0DqzR1O5xmKdu8sDEb3Rz1nl1qGsA4 afq8PKQuFxB7aj8awR6iMH/83uLqA6x6jyBof+jcdKKaAqBYHYKLIOX2mOQr7EHezmn1 NFb9iMXRPzx5lumHtSzrA/IO1havi4pwkr/6w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=cc:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer:sender; b=Wwc2LX7Cr5pVWbrpCgO7jYC/S6dWCcmMH7Q1RoHJLWURBUFcoGKF1X7FlNFLIhmU4/ TkxSPEowsD+wcaitz2K6qiUj0xEdQrosKPDzfxCzsJ8DjoFg/QOSNRTC2GeywbBuisdn nRUzeIx7/dkdE2QI9VXMSCZkt/jnRKlISPCJY= Received: by 10.210.104.20 with SMTP id b20mr16190398ebc.20.1225628605779; Sun, 02 Nov 2008 04:23:25 -0800 (PST) Received: from ?10.0.1.30? ([92.250.66.213]) by mx.google.com with ESMTPS id c24sm2995271ika.10.2008.11.02.04.23.23 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 02 Nov 2008 04:23:24 -0800 (PST) Message-Id: <8DE1D793-FA67-42D4-ACB9-36ECC41A5A39@fnop.net> From: Rui Paulo To: Tapac Khoma In-Reply-To: <4775ce300811020211v7c3a31e0p20d421a97c3e0fcf@mail.gmail.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v929.2) Date: Sun, 2 Nov 2008 12:23:20 +0000 References: <4775ce300811020211v7c3a31e0p20d421a97c3e0fcf@mail.gmail.com> X-Mailer: Apple Mail (2.929.2) Sender: Rui Paulo X-Mailman-Approved-At: Sun, 02 Nov 2008 13:18:49 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: FreeBSD development X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Nov 2008 12:23:27 -0000 On 2 Nov 2008, at 10:11, Tapac Khoma wrote: > Hello! > > I use FreeBSD 6.1 as my desktop system. I like it very much. But > there are > some features I want my system have. > I began to investigate of kernel structure and tried to write some > devices > drivers. > So I want to know can I help for you in development of my favor > system. And > where I can find useful information about writing device drivers under > FreeBSD. Some pointers: http://www.freebsd.org/projects/index.html http://www.freebsd.org/doc/en/books/developers-handbook/ http://www.freebsd.org/doc/en/books/arch-handbook/ Regards, -- Rui Paulo From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 2 13:35:19 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16C8A1065676 for ; Sun, 2 Nov 2008 13:35:19 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay01.ispgateway.de (smtprelay01.ispgateway.de [80.67.18.13]) by mx1.freebsd.org (Postfix) with ESMTP id C77D18FC16 for ; Sun, 2 Nov 2008 13:35:18 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from [88.153.0.16] (helo=localhost) by smtprelay01.ispgateway.de with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.68) (envelope-from ) id 1Kwcsa-0001Ut-6K for freebsd-hackers@freebsd.org; Sun, 02 Nov 2008 14:20:32 +0100 Date: Sun, 2 Nov 2008 14:20:26 +0100 From: Fabian Keil To: freebsd-hackers@freebsd.org Message-ID: <20081102142026.3d0057f0@fabiankeil.de> In-Reply-To: <20081026060906.GA66894@underworld.novel.ru> References: <20081026060906.GA66894@underworld.novel.ru> X-Mailer: Claws Mail 3.5.0 (GTK+ 2.12.11; i386-portbld-freebsd8.0) X-PGP-KEY-URL: http://www.fabiankeil.de/gpg-keys/freebsd-listen-2008-08-18.asc Mime-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/Je2NfIQl7E=yqhyZ+_9yuv."; protocol="application/pgp-signature"; micalg=PGP-SHA1 X-Df-Sender: 775067 Subject: Re: problems obtaining kernel dump X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Nov 2008 13:35:19 -0000 --Sig_/Je2NfIQl7E=yqhyZ+_9yuv. Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Roman Bogorodskiy wrote: > I'm having a problem obtaining kernel dump. The box has 512Mb of RAM. >=20 > In rc.conf I have: >=20 > dumpdev=3D"/dev/ad4s1b" > dumpdir=3D"/var/crash" >=20 > swapinfo -h gives the following: >=20 > Device 1K-blocks Used Avail Capacity > /dev/ad4s1b 1048576 0B 1.0G 0% >=20 > /var/crash directory exists, and the root partition where it is placed > has enough space as well: >=20 > /dev/ad4s1a 989M 350M 560M 38% / >=20 > The box runs fresh FreeBSD/i386 -CURRENT. >=20 > So, I do swapoff and then perform actions to reproduce the crash and it=20 > breaks me into ddb prompt. >=20 > I do: >=20 > call doadump > continue >=20 > it reboots, and when I run "savecore -v /var/crash /dev/ad4s1b" it > prints: >=20 > unable to open bounds file, using 0 > checking for kernel dump on device /dev/ad4s1b > mediasize =3D 1073741824 > sectorsize =3D 512 > magic mismatch on last dump header on /dev/ad4s1b > savecore: no dumps found >=20 > The same happens when I do 'panic' instead of 'call doadump' like > handbook suggests. What am I doing wrong? I googled for similar > problems, found some mail threads but didn't find meaningful advises > though. I'm seeing a similar problem with an amd64 system on a SATA disk running CURRENT from 20080727 (can't update as I'm using Pawel's ZFS patch and upgraded most of the zpools). Calling panic leads to an instant reboot, nothing seems to be dumped. Fabian --Sig_/Je2NfIQl7E=yqhyZ+_9yuv. Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) iEYEARECAAYFAkkNqRoACgkQBYqIVf93VJ1AawCdGpZey/LXNREMu2jbxoEYH6J9 1oAAoLHRnbK7r93LzvVkDfCatm0F59ip =5nzl -----END PGP SIGNATURE----- --Sig_/Je2NfIQl7E=yqhyZ+_9yuv.-- From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 2 15:06:32 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BC2B7106564A for ; Sun, 2 Nov 2008 15:06:32 +0000 (UTC) (envelope-from patfbsd@davenulle.org) Received: from smtp.lamaiziere.net (net.lamaiziere.net [91.121.44.19]) by mx1.freebsd.org (Postfix) with ESMTP id 7BD268FC1E for ; Sun, 2 Nov 2008 15:06:32 +0000 (UTC) (envelope-from patfbsd@davenulle.org) Received: from baby-jane.lamaiziere.net (94.10.87-79.rev.gaoland.net [79.87.10.94]) by smtp.lamaiziere.net (Postfix) with ESMTPA id 8290E633679; Sun, 2 Nov 2008 15:47:57 +0100 (CET) Received: from baby-jane-lamaiziere-net.local (localhost [127.0.0.1]) by baby-jane.lamaiziere.net (Postfix) with ESMTP id 556C46418AA; Sun, 2 Nov 2008 15:47:56 +0100 (CET) Date: Sun, 2 Nov 2008 15:47:55 +0100 From: Patrick =?ISO-8859-15?Q?Lamaizi=E8re?= To: "Tapac Khoma" Message-ID: <20081102154755.1504da7a@baby-jane-lamaiziere-net.local> In-Reply-To: <4775ce300811020211v7c3a31e0p20d421a97c3e0fcf@mail.gmail.com> References: <4775ce300811020211v7c3a31e0p20d421a97c3e0fcf@mail.gmail.com> Organization: /dave/nulle X-Mailer: Claws Mail 3.3.1 (GTK+ 2.12.9; i386-apple-darwin9.3.0) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 8bit Cc: freebsd-hackers@freebsd.org Subject: Re: FreeBSD development X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Nov 2008 15:06:32 -0000 Le Sun, 2 Nov 2008 12:11:19 +0200, "Tapac Khoma" a écrit : > Hello! > > I use FreeBSD 6.1 as my desktop system. I like it very much. But > there are some features I want my system have. > I began to investigate of kernel structure and tried to write some > devices drivers. > So I want to know can I help for you in development of my favor > system. And where I can find useful information about writing device > drivers under FreeBSD. I've found this paper quite useful. http://caia.swin.edu.au/reports/070622A/CAIA-TR-070622A.pdf Regards. From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 2 19:48:40 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 283B610656B5 for ; Sun, 2 Nov 2008 19:48:40 +0000 (UTC) (envelope-from shilp.kamal@yahoo.com) Received: from n69.bullet.mail.sp1.yahoo.com (n69.bullet.mail.sp1.yahoo.com [98.136.44.41]) by mx1.freebsd.org (Postfix) with SMTP id 0C7598FC2E for ; Sun, 2 Nov 2008 19:48:40 +0000 (UTC) (envelope-from shilp.kamal@yahoo.com) Received: from [216.252.122.219] by n69.bullet.mail.sp1.yahoo.com with NNFMP; 02 Nov 2008 19:35:05 -0000 Received: from [69.147.65.167] by t4.bullet.sp1.yahoo.com with NNFMP; 02 Nov 2008 19:35:05 -0000 Received: from [127.0.0.1] by omp502.mail.sp1.yahoo.com with NNFMP; 02 Nov 2008 19:35:05 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 320294.25417.bm@omp502.mail.sp1.yahoo.com Received: (qmail 70349 invoked by uid 60001); 2 Nov 2008 19:35:04 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type:Message-ID; b=VNesvuXlpsmFl4QVhQfgwXvCj99j7zVKOVpZ84pTH7v9E0w+8VHoYrh+H8PDWAyyFIXZpZ6VF2GO0DkS/j4UKipbXvxXd9c3FKQ1s6FmToNSNU2pNrcE/kFVH5TsR6QMvZvlx++YnEotdLUo0qTKCj9A4JQ8I/0tDAPBJSHTkSI=; X-YMail-OSG: 859YORQVM1miUU24BKcblYCfSJlffoLYOtEkjm0RzyT2NGyZPnUt54hjimxkbF2vNZEIGnr.dS1P_gaFDR7g1yN488SCHvhXydC56EAZd65jVtF23GguWu0V9X_xVLs5ciT2Xg-- Received: from [130.86.201.70] by web45407.mail.sp1.yahoo.com via HTTP; Sun, 02 Nov 2008 11:35:04 PST X-Mailer: YahooMailWebService/0.7.247.3 Date: Sun, 2 Nov 2008 11:35:04 -0800 (PST) From: Kamlesh Patel To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Message-ID: <781226.70062.qm@web45407.mail.sp1.yahoo.com> X-Mailman-Approved-At: Sun, 02 Nov 2008 21:48:38 +0000 Subject: Virtual Memory Improvement Algorithm X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: shilp.kamal@yahoo.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Nov 2008 19:48:40 -0000 Hi friends, My name is Kamlesh and i am working on freeBSD Virtual Memory Improvement Algorithm as my Master Project at California State University Sacramento. Could any one help me in Virtual Memory free space management: Splay tree to Radix tree? Thanks Kamlesh From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 2 22:40:06 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 420FC1065674 for ; Sun, 2 Nov 2008 22:40:06 +0000 (UTC) (envelope-from talon@lpthe.jussieu.fr) Received: from shiva.jussieu.fr (shiva.jussieu.fr [134.157.0.129]) by mx1.freebsd.org (Postfix) with ESMTP id C77DE8FC0C for ; Sun, 2 Nov 2008 22:40:05 +0000 (UTC) (envelope-from talon@lpthe.jussieu.fr) Received: from parthe.lpthe.jussieu.fr (parthe.lpthe.jussieu.fr [134.157.10.1]) by shiva.jussieu.fr (8.14.3/jtpda-5.4) with ESMTP id mA2Me3bl093810 for ; Sun, 2 Nov 2008 23:40:04 +0100 (CET) X-Ids: 168 Received: from niobe.lpthe.jussieu.fr (niobe.lpthe.jussieu.fr [134.157.10.41]) by parthe.lpthe.jussieu.fr (Postfix) with ESMTP id 9FC708A2A3 for ; Sun, 2 Nov 2008 23:40:02 +0100 (CET) Received: by niobe.lpthe.jussieu.fr (Postfix, from userid 2005) id A452414; Sun, 2 Nov 2008 23:40:01 +0100 (CET) Date: Sun, 2 Nov 2008 23:40:01 +0100 From: Michel Talon To: freebsd-hackers@freebsd.org Message-ID: <20081102224001.GA28831@lpthe.jussieu.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.0 (shiva.jussieu.fr [134.157.0.168]); Sun, 02 Nov 2008 23:40:04 +0100 (CET) X-Virus-Scanned: ClamAV 0.93.3/8555/Sun Nov 2 21:58:39 2008 on shiva.jussieu.fr X-Virus-Status: Clean X-Miltered: at jchkmail.jussieu.fr with ID 490E2C43.00C by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 490E2C43.00C/134.157.10.1/parthe.lpthe.jussieu.fr/parthe.lpthe.jussieu.fr/ X-j-chkmail-Score: MSGID : 490E2C43.00C on jchkmail.jussieu.fr : j-chkmail score : . : R=. U=. O=. B=0.075 -> S=0.075 X-j-chkmail-Status: Ham X-Mailman-Approved-At: Sun, 02 Nov 2008 22:56:42 +0000 Subject: Re: Virtual Memory Improvement Algorithm X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Nov 2008 22:40:06 -0000 Kamlesh Patel wrote: > My name is Kamlesh and i am working on freeBSD Virtual Memory > Improvement Algorithm as my Master Project at California State > University Sacramento. > > Could any one help me in Virtual Memory free space management: Splay > tree to Radix tree? You are a late comer, see: http://wiki.freebsd.org/MayurShardul/VM_Algorithm_Improvement -- Michel TALON From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 3 00:42:40 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6685F106564A for ; Mon, 3 Nov 2008 00:42:40 +0000 (UTC) (envelope-from prvs=julian=186a09c8d@elischer.org) Received: from smtp-outbound.ironport.com (smtp-outbound.ironport.com [63.251.108.112]) by mx1.freebsd.org (Postfix) with ESMTP id 52F9B8FC17 for ; Mon, 3 Nov 2008 00:42:40 +0000 (UTC) (envelope-from prvs=julian=186a09c8d@elischer.org) Received: from unknown (HELO julian-mac.elischer.org) ([10.251.60.35]) by smtp-outbound.ironport.com with ESMTP; 02 Nov 2008 16:14:06 -0800 Message-ID: <490E424D.4010509@elischer.org> Date: Sun, 02 Nov 2008 16:14:05 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.17 (Macintosh/20080914) MIME-Version: 1.0 To: shilp.kamal@yahoo.com References: <781226.70062.qm@web45407.mail.sp1.yahoo.com> In-Reply-To: <781226.70062.qm@web45407.mail.sp1.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: Virtual Memory Improvement Algorithm X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Nov 2008 00:42:40 -0000 Kamlesh Patel wrote: > Hi friends, > > My name is Kamlesh and i am working on freeBSD Virtual Memory Improvement > Algorithm as my Master Project at California State University Sacramento. > > Could any one help me in Virtual Memory free space management: Splay tree > to Radix tree? > > > Thanks > Kamlesh I suggest you talk with professor Cox (alc@freebsd.org) as he is the VM maintainer. > > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 3 09:32:16 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9F0D1065675 for ; Mon, 3 Nov 2008 09:32:16 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from shell.rawbw.com (shell.rawbw.com [198.144.192.42]) by mx1.freebsd.org (Postfix) with ESMTP id AFFB48FC13 for ; Mon, 3 Nov 2008 09:32:16 +0000 (UTC) (envelope-from yuri@rawbw.com) Received: from eagle.syrec.org (c-67-188-126-36.hsd1.ca.comcast.net [67.188.126.36]) (authenticated bits=0) by shell.rawbw.com (8.13.6/8.13.6) with ESMTP id mA39Y0x5070860 for ; Mon, 3 Nov 2008 01:34:00 -0800 (PST) Message-ID: <490EC51E.4070403@rawbw.com> Date: Mon, 03 Nov 2008 01:32:14 -0800 From: Yuri User-Agent: Thunderbird 2.0.0.17 (X11/20081001) MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: How to troubleshoot wireless problem? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: yuri@rawbw.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Nov 2008 09:32:16 -0000 My FreeBSD is able to connect to WiFi WEP network 95% of time. But 5% of time either ifconfig shown "not associated" status or dhclient fails to get IP address or name resolution isn't functioning after DHCP exchange is complete. Signal strength is -80:-95. After a while I am able to connect again. This happens with both ath0 and ral0 devices. In the meantime Linux box sitting next to FreeBSD box has no problem connecting to the same WiFi network at all times. This makes me think that something in FreeBSD isn't working right. How can I troubleshoot this problem? How stable is support for wireless networking in FreeBSD? I use this command line: ifconfig wepmode on ssid my-ssid weptxkey 1 wepkey 0xMYKEY dhcp on 7.1-PRERELEASE Yuri From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 3 13:11:02 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 98ABD1065674 for ; Mon, 3 Nov 2008 13:11:02 +0000 (UTC) (envelope-from onemda@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.247]) by mx1.freebsd.org (Postfix) with ESMTP id 4A8DB8FC1B for ; Mon, 3 Nov 2008 13:11:02 +0000 (UTC) (envelope-from onemda@gmail.com) Received: by an-out-0708.google.com with SMTP id b6so14418ana.13 for ; Mon, 03 Nov 2008 05:11:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=aKGoC6LMojy8k+fR5nek5aKVzl/D22XZtfVA9XZdj2Y=; b=fJZ5+AP142sIbRMa46GQ8JLGGNhDiNks60p8jCI+XOB/pHC1vdEjMIFvVpaLGN0+d/ oeXrjkZoGEs2CoYu7/surNoF19y4ge5AJk9T3N8hpeGQ71bwMLtkE+iaKOPR8tfULh2E XaAWHhBsLdYbwEje66BwSlAWyjz8g+pERbe+Q= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=xIFzT4HEIAf5Aapkogh1goscTu5TSknKOGLdMEatzlxVzl6gWzhoGu5SfluZFprg3L kZNpbLPHp1Nv4YXLrwkOyE0YVVDsSvFmh/4pr86UkHKn68qPpd/PPe3vguDCfKS8m0/n bda+jMi+Fvho2NbEIOkW144LbznIZb9Zk7kGQ= Received: by 10.231.15.205 with SMTP id l13mr488iba.14.1225716398521; Mon, 03 Nov 2008 04:46:38 -0800 (PST) Received: by 10.231.11.72 with HTTP; Mon, 3 Nov 2008 04:46:38 -0800 (PST) Message-ID: <3a142e750811030446g47d8658cgd6d3cd0daba19c58@mail.gmail.com> Date: Mon, 3 Nov 2008 13:46:38 +0100 From: "Paul B. Mahol" To: yuri@rawbw.com In-Reply-To: <490EC51E.4070403@rawbw.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <490EC51E.4070403@rawbw.com> Cc: freebsd-hackers@freebsd.org Subject: Re: How to troubleshoot wireless problem? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Nov 2008 13:11:02 -0000 On 11/3/08, Yuri wrote: > My FreeBSD is able to connect to WiFi WEP network 95% of time. > But 5% of time either ifconfig shown "not associated" status or dhclient > fails to get IP address or name resolution isn't functioning > after DHCP exchange is complete. Signal strength is -80:-95. > After a while I am able to connect again. > This happens with both ath0 and ral0 devices. > > In the meantime Linux box sitting next to FreeBSD box has > no problem connecting to the same WiFi network at all times. > > This makes me think that something in FreeBSD isn't working right. > > How can I troubleshoot this problem? > How stable is support for wireless networking in FreeBSD? > > I use this command line: > ifconfig wepmode on ssid my-ssid weptxkey 1 wepkey 0xMYKEY dhcp > on 7.1-PRERELEASE In wlan(4) it is explained in detail how to enable debuging. Also every driver have its own ways to toggle debug options. Note that wlan and driver modules must be compiled with debug enabled(at least it works in that way on CURRENT). Look in NOTES for explanation. From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 3 20:50:25 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2F0CD1065688 for ; Mon, 3 Nov 2008 20:50:25 +0000 (UTC) (envelope-from patfbsd@davenulle.org) Received: from smtp.lamaiziere.net (net.lamaiziere.net [91.121.44.19]) by mx1.freebsd.org (Postfix) with ESMTP id E393A8FC1E for ; Mon, 3 Nov 2008 20:50:24 +0000 (UTC) (envelope-from patfbsd@davenulle.org) Received: from baby-jane.lamaiziere.net (94.10.87-79.rev.gaoland.net [79.87.10.94]) by smtp.lamaiziere.net (Postfix) with ESMTPA id 28DB9633676 for ; Mon, 3 Nov 2008 21:50:23 +0100 (CET) Received: from baby-jane-lamaiziere-net.local (localhost [127.0.0.1]) by baby-jane.lamaiziere.net (Postfix) with ESMTP id A5D91642DD3 for ; Mon, 3 Nov 2008 21:50:22 +0100 (CET) Date: Mon, 3 Nov 2008 21:50:21 +0100 From: Patrick =?ISO-8859-15?Q?Lamaizi=E8re?= To: freebsd-hackers@freebsd.org Message-ID: <20081103215021.0412b7b2@baby-jane-lamaiziere-net.local> Organization: /dave/nulle X-Mailer: Claws Mail 3.3.1 (GTK+ 2.12.9; i386-apple-darwin9.3.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: problem with bus_dmamap_load_uio X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Nov 2008 20:50:25 -0000 Hello, (8-Current/i386) I've got a problem with bus_dmamap_load_uio(9). I want to use it to map an uio for DMA with the Geode security block (glxsb(4)). It works, i can make milions of crypto operations with cryptotest. But when (i guess, i'm not sure) the free memory becomes low, bus_dmamap_load_uio() fails with errno == EFBIG. I can reproduce the problem by running "periodic daily", "dd if=/dev/zero of=foo", ... By sample i run into problems with Mem: 33M Active, 310M Inact, 82M Wired, 60M Buf, 69M Free. When it fails the uio is : uio_segflg = UIO_SYSSPACE uio_iovcnt = 1, totlen = 16384 (the total length for iov) uio_resid = 16384 (i've got some failure with size between 4000 and 16384) I don't understand why bus_dmamap_load_uio() cannot load the uio. Also when it fails, i copy the uio into a buffer and then a bus_dmamap_load() of the buffer always works. The dma tag is allocated with: bus_dma_tag_create(bus_get_dma_tag(sc->sc_dev),/* parent */ 16, 0, /* alignments, bounds */ BUS_SPACE_MAXADDR_32BIT, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ 16384, /* maxsize */ 1, /* nsegments */ 16384, /* maxsegsize */ BUS_DMA_ALLOCNOW, /* flags */ NULL, NULL, /* lockfunc, lockarg */ &sc->sc_dmat); The dma map is created just before the bus_dmamap_load_uio() and destroyed later. (source : http://user.lamaiziere.net/patrick/glxsb.c , see glxsb_crypto_encdec() ). I'm trying to understand and fix this problem for several days, so any idea will be very cool... thanks, regards. From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 3 22:12:35 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7CB1D1065678 for ; Mon, 3 Nov 2008 22:12:35 +0000 (UTC) (envelope-from bahamasfranks@gmail.com) Received: from yx-out-2324.google.com (yx-out-2324.google.com [74.125.44.30]) by mx1.freebsd.org (Postfix) with ESMTP id 22E898FC0C for ; Mon, 3 Nov 2008 22:12:34 +0000 (UTC) (envelope-from bahamasfranks@gmail.com) Received: by yx-out-2324.google.com with SMTP id 8so1014957yxb.13 for ; Mon, 03 Nov 2008 14:12:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:reply-to :sender:to:subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references :x-google-sender-auth; bh=LIk3NahC+xLuS/+MKq2Vg69YV94fSuuNejU8YUaiSbA=; b=KDoKweXHB6ayNsUxSKuUorp6lv/PmRCThZqH0aeS1gnrzRgREIUVnokH3s6Xkqo+5b vLLtyjKavYS9h/ZX4N+xp+JrYYmGsvWxAJWQ1DZ0uB2uyEatCJumoy1EJSyzf3CWsarJ tZQXo2QyCuHytoQY1g+WfXBbBLeJ/oVg/6htw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:reply-to:sender:to:subject:cc:in-reply-to :mime-version:content-type:content-transfer-encoding :content-disposition:references:x-google-sender-auth; b=VS6aqCEBrXmoCP5/NeAhaKcdVBPCeVI9NwWFtwrg7rY/iVxrdt0d9nRg4nFjvx+1xd bzPoQaLFe7IcmhiJBDG6Jfn8zCn9JwqlOnSsuXEo184Ms+yBwhMzX3I3JuvDhatXNbGV ndStz1IEtCwh+ZDv/+J6unJUdUInrO4lI5GDc= Received: by 10.100.95.19 with SMTP id s19mr266224anb.43.1225750354139; Mon, 03 Nov 2008 14:12:34 -0800 (PST) Received: by 10.100.132.8 with HTTP; Mon, 3 Nov 2008 14:12:34 -0800 (PST) Message-ID: <539c60b90811031412y35e2599cneca5912ada43b081@mail.gmail.com> Date: Mon, 3 Nov 2008 15:12:34 -0700 From: "Steve Franks" Sender: bahamasfranks@gmail.com To: "Giorgos Keramidas" In-Reply-To: <87r65vl9ur.fsf@kobe.laptop> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <539c60b90810311123w2aa94b8akcd0a5d0fe791885a@mail.gmail.com> <539c60b90810311230i11460966la7ff35b0093642ec@mail.gmail.com> <87r65vl9ur.fsf@kobe.laptop> X-Google-Sender-Auth: 6c469db531fc2e05 Cc: freebsd-hackers Subject: Re: includes, configure, /usr/lib vs. /usr/local/lib, and linux coders X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: stevefranks@ieee.org List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Nov 2008 22:12:35 -0000 That's alot of good info. It should go in the porter's handbook, maybe... So, if I'm looking to make a port, which one of those people should I be acting as? Maintainer? That's FreeBSD-port-terminology you are using, correct? Steve On Sat, Nov 1, 2008 at 8:55 AM, Giorgos Keramidas wrote: > On Fri, 31 Oct 2008 12:30:46 -0700, "Steve Franks" wrote: >> Let's backup. What's the 'right' way to get a bloody linux program >> that expects all it's headers in /usr/include to compile on freebsd >> where all the headers are in /usr/local/include? That's all I'm >> really asking. Specifically, it's looking for libusb & libftdi. If I >> just type gmake, it can't find it, but if I manually edit the >> Makefiles to add -I/usr/local/include, it can. Obviously, manually >> editing the makefiles is *not* the right way to fix it (plus it's >> driving me crazy). > > Then you run `configure' with the `right' environment: > > env CPPFLAGS='-I/usr/local/include' \ > LDFLAGS='-L/usr/local/lib' ./configure > > The `--includedir' and `--libdir' options are *not* meant to be useful > to the developer that uses the GNU build tools, but to the person who > compiles something for the target host. > > There are several types of people involved in the `release' of a full > program: > > * The maintainer, who uses `automake', `libtool' and `autoconf' to > write portable Makefiles and build tools. > > * The builder, who compiles the program with specific options. > > * The packager, who runs `make install' in the build tree, creates a > set of installed files, and then packages *some* of these files in > a packaging-specific format. > > These types of people commonly have different constraints in the way > they can tweak and twist the GNU build tools, to match their needs. > > 1. The _maintainer_ of the program is free to set up his environment to > include any `CPPFLAGS', `CFLAGS' or `LDFLAGS' they find useful. For > example, if they have an experimental third-party library installed > in a custom location they can use: > > export CPPFLAGS='-I/opt/foolib/include' LDFLAGS='-L/opt/foolib/lib' > ./configure --prefix='/var/tmp/myprog' > > This way `configure' will emit Makefiles that try to use the > third-party library from `/opt/foolib' instead of the system-default > location for header files and libraries. > > This may often be a lot easier than waiting until the necessary bits > are installed in the ``official'' places at development systems. > Developers who want to experiment with a new version of `libfoo', > i.e. to integrate it with the rest of a program, can use custom > `CPPFLAGS' and `LDFLAGS' while everyone else is merrily going along > with the ``standard'' version of the `libfoo' library. > > 2. The _builder_ may be constrained in the sets of options he can pass > to the `CFLAGS'. He is, after all, testing how a pristine, clean > version of the program can build in what is defined as the ``official > release'' environment. > > He may be allowed to tinker with include paths and library paths, but > it is often safer to wrap the build process in scripts and tools that > yield a repeatable, verifiable build process. This may preclude the > use of options like `-I/custom/hdr/path' and `-L/custom/lib/path'. > > The builder of a program may not be an actual person, but a cron job > or another automated process, i.e. a `nightly build' script that runs > a clean build in a pristine environment, verifies that it can all > complete without errors, and then emails one or more reports. > > When the builder _is_ a real person, he may be sharing roles with the > third type of person involved in the build life of a program that > uses the GNU build tools: the packaging person. > > 3. The _packager_ is someone who runs `make install', to produce the > final program distribution and then bundles parts of or even all the > files that are produced by the usual `install' target of GNU tools. > The installation of all the files may be done in the default > installation `prefix', but it may also be redirected to a staging > area by setting `DESTDIR' in the environment: > > mkdir /var/tmp/proto > env DESTDIR=/var/tmp/proto make install > > Depending on the type of the target system, and on particular needs > of the packaging person, there may be cases where certain files have > to be installed in a `non-standard' location, or in a location that > was not foreseen by the original maintainer. In that case, the > packager can use the `--libdir' and `--includedir' options to change > the final, installed location of the relevant bits. > > A typical example is the case of Solaris systems, where libraries may > be installed in `/usr/lib/64' for 64-bit architectures. When a > packager prepares installation images for these architectures, he can > build the program with: > > ./configure --prefix='/opt/foo' --libdir='$prefix/lib/64' > > All this is a pretty long-winded way of saying: > > The `--includedir' and `--libdir' options are not really something > that is meant to be a convenience option for the _maintainer_ of a > program,, the person who writes the code. They are meant to be > useful tools for the _packager_ of the program, the person who > builds and prepares the final, install-able images. > > If you are the maintainer, who writes the code of a program, and you > find yourself in a position where you need to use the `--libdir' and > the `--includedir' options, then YOU ARE DOING IT WRONG. > > and, as you have probably guessed by now: > > The Linux people of the original post are wrong. > > The GNU build tools are a relatively good set of tools to automate a lot > of the mundane details of setting up a build system. It is unfortunate > that many people who use them have absolutely no clue of what they are > using, and they think that slapping a bunch of copy-paste snippets from > Google searches in a `configure.in' script will magically turn any odd > mess into a clean release process. Alas, Linux newbies are very often > exactly _this_ sort of person :/ > > From owner-freebsd-hackers@FreeBSD.ORG Mon Nov 3 22:51:25 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A2581065672 for ; Mon, 3 Nov 2008 22:51:25 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 7B0898FC21 for ; Mon, 3 Nov 2008 22:51:24 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [IPv6:::1]) (authenticated bits=0) by server.baldwin.cx (8.14.3/8.14.3) with ESMTP id mA3MpFpl093686; Mon, 3 Nov 2008 17:51:15 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Mon, 3 Nov 2008 18:49:57 -0500 User-Agent: KMail/1.9.7 References: <20081103215021.0412b7b2@baby-jane-lamaiziere-net.local> In-Reply-To: <20081103215021.0412b7b2@baby-jane-lamaiziere-net.local> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <200811031849.57232.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [IPv6:::1]); Mon, 03 Nov 2008 17:51:15 -0500 (EST) X-Virus-Scanned: ClamAV 0.93.1/8560/Mon Nov 3 14:20:13 2008 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00,NO_RELAYS autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Patrick =?iso-8859-1?q?Lamaizi=E8re?= Subject: Re: problem with bus_dmamap_load_uio X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Nov 2008 22:51:25 -0000 On Monday 03 November 2008 03:50:21 pm Patrick Lamaizi=E8re wrote: > Hello, >=20 > (8-Current/i386) >=20 > I've got a problem with bus_dmamap_load_uio(9). I want to use it to > map an uio for DMA with the Geode security block (glxsb(4)). >=20 > It works, i can make milions of crypto operations with cryptotest. >=20 > But when (i guess, i'm not sure) the free memory becomes low, > bus_dmamap_load_uio() fails with errno =3D=3D EFBIG. >=20 > I can reproduce the problem by running "periodic daily", "dd > if=3D/dev/zero of=3Dfoo", ... >=20 > By sample i run into problems with Mem: 33M Active, 310M Inact, 82M > Wired, 60M Buf, 69M Free. >=20 > When it fails the uio is : > uio_segflg =3D UIO_SYSSPACE > uio_iovcnt =3D 1,=20 > totlen =3D 16384 (the total length for iov) > uio_resid =3D 16384 >=20 > (i've got some failure with size between 4000 and 16384) >=20 > I don't understand why bus_dmamap_load_uio() cannot load the uio. Also > when it fails, i copy the uio into a buffer and then a > bus_dmamap_load() of the buffer always works. >=20 > The dma tag is allocated with: >=20 > bus_dma_tag_create(bus_get_dma_tag(sc->sc_dev),/* parent */ > 16, 0, /* alignments, bounds */ > BUS_SPACE_MAXADDR_32BIT, /* lowaddr */ > BUS_SPACE_MAXADDR, /* highaddr */ > NULL, NULL, /* filter, filterarg */ > 16384, /* maxsize */ > 1, /* nsegments */ > 16384, /* maxsegsize */ > BUS_DMA_ALLOCNOW, /* flags */ > NULL, NULL, /* lockfunc, lockarg */ > &sc->sc_dmat); >=20 > The dma map is created just before the bus_dmamap_load_uio() and > destroyed later. >=20 > (source : http://user.lamaiziere.net/patrick/glxsb.c , see > glxsb_crypto_encdec() ). >=20 > I'm trying to understand and fix this problem for several days, so any > idea will be very cool... Your dma tag only allows a single scatter/gather entry (nsegments). What i= s=20 happening is that under memory pressure, the virtual buffer in userland is= =20 not built from physically contiguous pages, so you would need multiple s/g= =20 entries to describe the buffer. Hence EFBIG. If your hardware can handle= =20 multiple s/g entries, then just increase 'nsegments' and handle the multipl= e=20 segments in your callback routine. =2D-=20 John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 4 12:34:14 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6CC8C1065676 for ; Tue, 4 Nov 2008 12:34:14 +0000 (UTC) (envelope-from patfbsd@davenulle.org) Received: from smtp.lamaiziere.net (net.lamaiziere.net [91.121.44.19]) by mx1.freebsd.org (Postfix) with ESMTP id 206958FC1B for ; Tue, 4 Nov 2008 12:34:14 +0000 (UTC) (envelope-from patfbsd@davenulle.org) Received: from baby-jane.lamaiziere.net (136.10.87-79.rev.gaoland.net [79.87.10.136]) by smtp.lamaiziere.net (Postfix) with ESMTPA id 53750633679; Tue, 4 Nov 2008 13:34:12 +0100 (CET) Received: from baby-jane-lamaiziere-net.local (localhost [127.0.0.1]) by baby-jane.lamaiziere.net (Postfix) with ESMTP id 168DF6434B3; Tue, 4 Nov 2008 13:34:10 +0100 (CET) Date: Tue, 4 Nov 2008 13:34:08 +0100 From: Patrick =?ISO-8859-15?Q?Lamaizi=E8re?= To: John Baldwin Message-ID: <20081104133408.7a010b63@baby-jane-lamaiziere-net.local> In-Reply-To: <200811031849.57232.jhb@freebsd.org> References: <20081103215021.0412b7b2@baby-jane-lamaiziere-net.local> <200811031849.57232.jhb@freebsd.org> Organization: /dave/nulle X-Mailer: Claws Mail 3.3.1 (GTK+ 2.12.9; i386-apple-darwin9.3.0) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 8bit Cc: freebsd-hackers@freebsd.org Subject: Re: problem with bus_dmamap_load_uio X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Nov 2008 12:34:14 -0000 Le Mon, 3 Nov 2008 18:49:57 -0500, John Baldwin a écrit : Hello, > > I don't understand why bus_dmamap_load_uio() cannot load the uio. > > Also when it fails, i copy the uio into a buffer and then a > > bus_dmamap_load() of the buffer always works. > > > > The dma tag is allocated with: > > > > bus_dma_tag_create(bus_get_dma_tag(sc->sc_dev),/* parent */ > > 16, 0, /* alignments, bounds */ > > BUS_SPACE_MAXADDR_32BIT, /* lowaddr */ > > BUS_SPACE_MAXADDR, /* highaddr */ > > NULL, NULL, /* filter, filterarg */ > > 16384, /* maxsize */ > > 1, /* nsegments */ > > 16384, /* maxsegsize */ > > BUS_DMA_ALLOCNOW, /* flags */ > > NULL, NULL, /* lockfunc, lockarg */ > > &sc->sc_dmat); ... > Your dma tag only allows a single scatter/gather entry (nsegments). > What is happening is that under memory pressure, the virtual buffer > in userland is not built from physically contiguous pages, so you > would need multiple s/g entries to describe the buffer. Hence > EFBIG. Thank you John, you save me! > If your hardware can handle multiple s/g entries, then just > increase 'nsegments' and handle the multiple segments in your > callback routine. Unfortunely it does not. Regards. From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 4 19:04:42 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE8921065673 for ; Tue, 4 Nov 2008 19:04:42 +0000 (UTC) (envelope-from venkatraman_t@yahoo.com) Received: from web34406.mail.mud.yahoo.com (web34406.mail.mud.yahoo.com [66.163.178.155]) by mx1.freebsd.org (Postfix) with SMTP id 5BAE58FC1B for ; Tue, 4 Nov 2008 19:04:42 +0000 (UTC) (envelope-from venkatraman_t@yahoo.com) Received: (qmail 84792 invoked by uid 60001); 4 Nov 2008 18:38:02 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Message-ID; b=PBHz0IjDmMHgFHbdUPbWeljwkQPKlHtd8DBMEqVP7dtHaqqvGSClC9HE1HCENDtrMsUxm0Up3X5CDpnfUeZn00tj/RIEc72bOB/QG2OErPcCfRlF/CZw6sji+F9U1UAnX32MbAWH60GawJWbIp6JvOupMQ4lk3zf3MHskPq+Yzw=; X-YMail-OSG: UMp6r2cVM1lvfc9yUM1RJry.efy3iF1LKMNbI6XIEghDgIMWe1sgpFBoTmnvVjxC1w-- Received: from [66.129.224.36] by web34406.mail.mud.yahoo.com via HTTP; Tue, 04 Nov 2008 10:38:02 PST X-Mailer: YahooMailRC/1155.20 YahooMailWebService/0.7.260.1 Date: Tue, 4 Nov 2008 10:38:02 -0800 (PST) From: Venkatraman T To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Message-ID: <379172.84312.qm@web34406.mail.mud.yahoo.com> X-Mailman-Approved-At: Tue, 04 Nov 2008 20:17:02 +0000 Subject: Using GCov in the FreeBSD kernel X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Nov 2008 19:04:42 -0000 Hi, I am trying to understand what how much effort it would take to get the code coverage in FreeBSD kernel using gcov. My understanding is we have gcov in FreeBSD which gets code coverage on user space programs. The reasons that this can't be used in kernel right now is because: 1) gcov libs cannot use the standard C libs when the kernel code is compiled with the gcov flags 2) There might be some issues with the kernel's access to the file systems for dumping the .gcda files What are the other issues that I need to look at? There is also a kernel patch for Linux (LTP) that allows gcov to get coverage in the kernel. Will it easier to patch FreeBSD gcov libraries to run on the kernel or should I try and modify the Linux kernel patch to work on FreeBSD kernel. I am very new to anything in the kernel and would appreciate any insight on how to go about doing this. Thanks Venkat From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 4 22:13:41 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD5C2106564A for ; Tue, 4 Nov 2008 22:13:41 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 442398FC1E for ; Tue, 4 Nov 2008 22:13:40 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1KxU9X-0005Em-LV for freebsd-hackers@freebsd.org; Tue, 04 Nov 2008 22:13:35 +0000 Received: from 93-138-69-155.adsl.net.t-com.hr ([93.138.69.155]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 04 Nov 2008 22:13:35 +0000 Received: from ivoras by 93-138-69-155.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 04 Nov 2008 22:13:35 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Tue, 04 Nov 2008 23:13:19 +0100 Lines: 64 Message-ID: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig46FEBD7A4700E2C477855B8A" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 93-138-69-155.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) X-Enigmail-Version: 0.95.7 Sender: news Subject: YAVP (Yet Another Vmstat Patch) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Nov 2008 22:13:41 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig46FEBD7A4700E2C477855B8A Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi, I'm working on something that tends to generate a lot of context switches and I don't like the way values in "vmstat 1" are printed practically unbounded, causing wrapping, etc. Here's a patch against -CURRENT that uses humanize_number() to keep the formatting of memory and context switches tabular in the common case: http://people.freebsd.org/~ivoras/diffs/vmstat.c.patch.txt The following people might not like it: * Those who really need to distinguish between 185400 context switches and 185499 context switches * Those who have scripts that parse the output * Those who thought the previous chaotic behaviour is cute :) Example of new output is: procs memory page disks faults cp= u r b w avm free flt re pi po fr sr ad4 ad8 in sy cs us sy id 3 0 0 112M 3.7G 0 0 0 0 0 0 0 0 2 373K 267K 4 37 59 2 0 0 112M 3.7G 0 0 0 0 0 0 0 0 0 366K 271K 4 37 59 3 0 0 112M 3.7G 0 0 0 0 0 0 0 0 0 370K 267K 4 36 59 4 0 0 112M 3.7G 0 0 0 0 0 0 0 0 0 378K 278K 5 36 60 3 0 0 112M 3.7G 0 0 0 0 0 0 0 0 69 366K 268K 4 39 58 2 0 0 112M 3.7G 0 0 0 0 0 0 0 0 65 372K 271K 5 41 54 4 0 0 112M 3.7G 0 0 0 0 0 0 0 0 71 374K 271K 5 40 55 2 0 0 112M 3.7G 0 0 0 0 0 0 0 0 6 375K 276K 4 39 57 Any objections to the patch? --------------enig46FEBD7A4700E2C477855B8A Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEUEARECAAYFAkkQyQgACgkQldnAQVacBcif5QCg6G7L6xXScsshYXn/Q3hFWL8I /74AmO7da9UVABzQmyVSRzxvuWSHQIM= =IRgo -----END PGP SIGNATURE----- --------------enig46FEBD7A4700E2C477855B8A-- From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 4 22:42:52 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68F141065688 for ; Tue, 4 Nov 2008 22:42:52 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.171]) by mx1.freebsd.org (Postfix) with ESMTP id 0B9768FC13 for ; Tue, 4 Nov 2008 22:42:52 +0000 (UTC) (envelope-from max@love2party.net) Received: from vampire.homelinux.org (dslb-088-066-002-145.pools.arcor-ip.net [88.66.2.145]) by mrelayeu.kundenserver.de (node=mrelayeu8) with ESMTP (Nemesis) id 0ML31I-1KxUbq2me1-0003Nb; Tue, 04 Nov 2008 23:42:51 +0100 Received: (qmail 21149 invoked from network); 4 Nov 2008 22:42:50 -0000 Received: from fbsd8.laiers.local (192.168.4.151) by ns1.laiers.local with SMTP; 4 Nov 2008 22:42:50 -0000 From: Max Laier Organization: FreeBSD To: freebsd-hackers@freebsd.org Date: Tue, 4 Nov 2008 23:42:49 +0100 User-Agent: KMail/1.10.1 (FreeBSD/8.0-CURRENT; KDE/4.1.1; i386; ; ) References: <200810071314.45922.max@love2party.net> <200811012114.43137.max@love2party.net> In-Reply-To: <200811012114.43137.max@love2party.net> MIME-Version: 1.0 Content-Type: Multipart/Mixed; boundary="Boundary-00=_p/MEJlOmDTp9+rV" Message-Id: <200811042342.49827.max@love2party.net> X-Provags-ID: V01U2FsdGVkX1+HHEGhk7qhy4pCfA85P25f8sTk/n9hTht8DZf 8K+EERiXawykj2HbLAVlAkpi5N1jZ+lVGZ3MruRK/05D2xUo/Z iRmMS9Xz9E5ktMnDOfptQ== Cc: lhmwzy@gmail.com, koitsu@freebsd.org, pjd@freebsd.org, Giorgos Keramidas , Pete French Subject: Re: du -A / -B options [Re: zfs quota question] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Nov 2008 22:42:52 -0000 --Boundary-00=_p/MEJlOmDTp9+rV Content-Type: text/plain; charset="iso-8859-15" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hi again, On Saturday 01 November 2008 21:14:42 I wrote: > a thread on freebsd-stable@ [1] about problems with du(1) and compressed > zfs filesystems got me looking for a possible solution. Attached is a di= ff > for du(1) that adds two new options: > > -A to display the apparent size of the file instead of the used blocks. > -B bsize to specify a custom blocksize. In particular one <512byte > > The GNU du(1) has --apparent-size for -A, but we don't like long options. > That's not to say that it couldn't be added for script compat. -B is > probably not that interesting, but it can be helpful and came for free. Attached is an updated patch. This refines the -B option to something more= =20 useful (and fixes a bug in the original patch). =46rom the man page: -B blocksize Calculate block counts in blocksize byte blocks. This is diff= er- ent from the -k, -m options or setting BLOCKSIZE and gives an estimate of how many space the examined file hierachy would require on a filesystem with the given blocksize. Unless in -A mode, blocksize is rounded up to the next multiple of 512. I think this is also very helpful and came for almost free. I made sure to= =20 not loose any range in the calculations when not in -A mode. In -A mode an= d=20 with small -B values you obviously can blast the 2^63bit block counter in=20 struct fts earlier. Please review, test & comment ... I plan to commit this in a few days if I= =20 don't hear otherwise. Thanks. > Any objections against the general concept? It's rather complicated to g= et > the apparent size of a directory hierarchy without scripting. I often > wonder if some hierarchy will fit on a CD/DVD and compressed zfs makes th= is > really difficult. > > As for the code, I know that there are a couple of style(9) errors in the= re > - mostly because the lines already exceeded 80 chars before my changes and > I plan to clean that up before I commit - should there be enough support > for the change itself. > > Thoughts? > > [1] > http://lists.freebsd.org/pipermail/freebsd-stable/2008-October/045698.html =2D-=20 /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --Boundary-00=_p/MEJlOmDTp9+rV Content-Type: text/plain; charset="iso-8859-15"; name="du.AB.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="du.AB.diff" Index: du.1 =================================================================== --- du.1 (revision 184563) +++ du.1 (working copy) @@ -40,11 +40,12 @@ .Nd display disk usage statistics .Sh SYNOPSIS .Nm +.Op Fl A .Op Fl H | L | P .Op Fl a | s | d Ar depth .Op Fl c .Op Fl l -.Op Fl h | k | m +.Op Fl h | k | m | B Ar blocksize .Op Fl n .Op Fl x .Op Fl I Ar mask @@ -60,6 +61,25 @@ .Pp The options are as follows: .Bl -tag -width indent +.It Fl A +Display the apparent size instead of the disk usage. +This can be helpful when operating on compressed volumes or sparse files. +.It Fl B Ar blocksize +Calculate block counts in +.Ar blocksize +byte blocks. +This is different from the +.Fl k, m +options or setting +.Ev BLOCKSIZE +and gives an estimate of how many space the examined file hierachy would +require on a filesystem with the given +.Ar blocksize . +Unless in +.Fl A +mode, +.Ar blocksize +is rounded up to the next multiple of 512. .It Fl H Symbolic links on the command line are followed, symbolic links in file hierarchies are not followed. @@ -136,14 +156,19 @@ If the environment variable .Ev BLOCKSIZE is set, and the -.Fl k -option is not specified, the block counts will be displayed in units of that -size block. +.Fl k, m +or +.Fl h +options are not specified, the block counts will be displayed in units of +that size block. If .Ev BLOCKSIZE is not set, and the -.Fl k -option is not specified, the block counts will be displayed in 512-byte blocks. +.Fl k, m +or +.Fl h +options are not specified, the block counts will be displayed in 512-byte +blocks. .El .Sh SEE ALSO .Xr df 1 , Index: du.c =================================================================== --- du.c (revision 184656) +++ du.c (working copy) @@ -79,14 +79,15 @@ static int ignorep(FTSENT *); static int nodumpflag = 0; +static int Aflag; +static long blocksize, cblocksize; int main(int argc, char *argv[]) { FTS *fts; FTSENT *p; - off_t savednumber; - long blocksize; + off_t savednumber, curblocks; int ftsoptions; int listall; int depth; @@ -98,16 +99,30 @@ setlocale(LC_ALL, ""); Hflag = Lflag = Pflag = aflag = sflag = dflag = cflag = hflag = - lflag = 0; + lflag = Aflag = 0; save = argv; ftsoptions = 0; savednumber = 0; + cblocksize = DEV_BSIZE; + blocksize = 0; depth = INT_MAX; SLIST_INIT(&ignores); - while ((ch = getopt(argc, argv, "HI:LPasd:chklmnrx")) != -1) + while ((ch = getopt(argc, argv, "AB:HI:LPasd:chklmnrx")) != -1) switch (ch) { + case 'A': + Aflag = 1; + break; + case 'B': + errno = 0; + cblocksize = atoi(optarg); + if (errno == ERANGE || cblocksize < 0) { + warnx("invalid argument to option B: %s", + optarg); + usage(); + } + break; case 'H': Hflag = 1; break; @@ -144,22 +159,18 @@ cflag = 1; break; case 'h': - if (setenv("BLOCKSIZE", "512", 1) == -1) - warn("setenv: cannot set BLOCKSIZE=512"); hflag = 1; break; case 'k': hflag = 0; - if (setenv("BLOCKSIZE", "1024", 1) == -1) - warn("setenv: cannot set BLOCKSIZE=1024"); + blocksize = 1024; break; case 'l': lflag = 1; break; case 'm': hflag = 0; - if (setenv("BLOCKSIZE", "1048576", 1) == -1) - warn("setenv: cannot set BLOCKSIZE=1048576"); + blocksize = 1048576; break; case 'n': nodumpflag = 1; @@ -206,6 +217,9 @@ if (Pflag) ftsoptions |= FTS_PHYSICAL; + if (!Aflag && (cblocksize % DEV_BSIZE) != 0) + cblocksize = howmany(cblocksize, DEV_BSIZE) * DEV_BSIZE; + listall = 0; if (aflag) { @@ -224,9 +238,14 @@ argv[1] = NULL; } - (void)getbsize(¬used, &blocksize); - blocksize /= 512; + if (blocksize == 0) + (void)getbsize(¬used, &blocksize); + if (!Aflag) { + cblocksize /= DEV_BSIZE; + blocksize /= DEV_BSIZE; + } + rval = 0; if ((fts = fts_open(argv, ftsoptions, NULL)) == NULL) @@ -242,17 +261,19 @@ if (ignorep(p)) break; + curblocks = Aflag ? + howmany(p->fts_statp->st_size, cblocksize) : + howmany(p->fts_statp->st_blocks, cblocksize); p->fts_parent->fts_bignum += p->fts_bignum += - p->fts_statp->st_blocks; + curblocks; if (p->fts_level <= depth) { if (hflag) { - prthumanval(howmany(p->fts_bignum, - blocksize)); + prthumanval(p->fts_bignum); (void)printf("\t%s\n", p->fts_path); } else { (void)printf("%jd\t%s\n", - (intmax_t)howmany(p->fts_bignum, + howmany(p->fts_bignum * cblocksize, blocksize), p->fts_path); } } @@ -273,21 +294,22 @@ linkchk(p)) break; + curblocks = Aflag ? + howmany(p->fts_statp->st_size, cblocksize) : + howmany(p->fts_statp->st_blocks, cblocksize); + if (listall || p->fts_level == 0) { if (hflag) { - prthumanval(howmany( - p->fts_statp->st_blocks, - blocksize)); + prthumanval(curblocks); (void)printf("\t%s\n", p->fts_path); } else { (void)printf("%jd\t%s\n", - (intmax_t)howmany( - p->fts_statp->st_blocks, - blocksize), p->fts_path); + howmany(curblocks * cblocksize, + blocksize), p->fts_path); } } - p->fts_parent->fts_bignum += p->fts_statp->st_blocks; + p->fts_parent->fts_bignum += curblocks; } savednumber = p->fts_parent->fts_bignum; } @@ -297,11 +319,11 @@ if (cflag) { if (hflag) { - prthumanval(howmany(savednumber, blocksize)); + prthumanval(savednumber); (void)printf("\ttotal\n"); } else { (void)printf("%jd\ttotal\n", (intmax_t)howmany( - savednumber, blocksize)); + savednumber * cblocksize, blocksize)); } } @@ -448,7 +470,9 @@ { char buf[5]; - bytes *= DEV_BSIZE; + bytes *= cblocksize; + if (!Aflag) + bytes *= DEV_BSIZE; humanize_number(buf, sizeof(buf), bytes, "", HN_AUTOSCALE, HN_B | HN_NOSPACE | HN_DECIMAL); @@ -460,8 +484,9 @@ usage(void) { (void)fprintf(stderr, - "usage: du [-H | -L | -P] [-a | -s | -d depth] [-c] " - "[-l] [-h | -k | -m] [-n] [-x] [-I mask] [file ...]\n"); + "usage: du [-A] [-H | -L | -P] [-a | -s | -d depth] [-c] " + "[-l] [-h | -k | -m | -B bsize] [-n] [-x] [-I mask] " + "[file ...]\n"); exit(EX_USAGE); } --Boundary-00=_p/MEJlOmDTp9+rV-- From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 00:05:56 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EF8F0106568D for ; Wed, 5 Nov 2008 00:05:55 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from igloo.linux.gr (igloo.linux.gr [62.1.205.36]) by mx1.freebsd.org (Postfix) with ESMTP id 6B4098FC0C for ; Wed, 5 Nov 2008 00:05:55 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from kobe.laptop (adsl92-78.kln.forthnet.gr [77.49.59.78]) (authenticated bits=128) by igloo.linux.gr (8.14.3/8.14.3/Debian-5) with ESMTP id mA505ToB003284 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 5 Nov 2008 02:05:36 +0200 Received: from kobe.laptop (kobe.laptop [127.0.0.1]) by kobe.laptop (8.14.3/8.14.3) with ESMTP id mA505TpO098843; Wed, 5 Nov 2008 02:05:29 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) Received: (from keramida@localhost) by kobe.laptop (8.14.3/8.14.3/Submit) id mA505Sjl098842; Wed, 5 Nov 2008 02:05:28 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) From: Giorgos Keramidas To: Ivan Voras In-Reply-To: (Ivan Voras's message of "Tue, 04 Nov 2008 23:13:19 +0100") Date: Wed, 05 Nov 2008 02:00:12 +0200 Message-ID: <87mygfkpo3.fsf@kobe.laptop> References: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-MailScanner-ID: mA505ToB003284 X-Hellug-MailScanner: Found to be clean X-Hellug-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-3.853, required 5, autolearn=not spam, ALL_TRUSTED -1.80, AWL 0.55, BAYES_00 -2.60) X-Hellug-MailScanner-From: keramida@ceid.upatras.gr X-Spam-Status: No Cc: freebsd-hackers@freebsd.org Subject: Re: YAVP (Yet Another Vmstat Patch) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 00:05:56 -0000 On Tue, 04 Nov 2008 23:13:19 +0100, Ivan Voras wrote: > Hi, > > I'm working on something that tends to generate a lot of context > switches and I don't like the way values in "vmstat 1" are printed > practically unbounded, causing wrapping, etc. > > Here's a patch against -CURRENT that uses humanize_number() to keep the > formatting of memory and context switches tabular in the common case: > > http://people.freebsd.org/~ivoras/diffs/vmstat.c.patch.txt > > The following people might not like it: > > * Those who really need to distinguish between 185400 context switches > and 185499 context switches > * Those who have scripts that parse the output > * Those who thought the previous chaotic behaviour is cute :) > > Example of new output is: > > procs memory page disks faults cpu > r b w avm free flt re pi po fr sr ad4 ad8 in sy cs us sy id > 3 0 0 112M 3.7G 0 0 0 0 0 0 0 0 2 373K 267K 4 37 59 > 2 0 0 112M 3.7G 0 0 0 0 0 0 0 0 0 366K 271K 4 37 59 > 3 0 0 112M 3.7G 0 0 0 0 0 0 0 0 0 370K 267K 4 36 59 > 4 0 0 112M 3.7G 0 0 0 0 0 0 0 0 0 378K 278K 5 36 60 > 3 0 0 112M 3.7G 0 0 0 0 0 0 0 0 69 366K 268K 4 39 58 > 2 0 0 112M 3.7G 0 0 0 0 0 0 0 0 65 372K 271K 5 41 54 > 4 0 0 112M 3.7G 0 0 0 0 0 0 0 0 71 374K 271K 5 40 55 > 2 0 0 112M 3.7G 0 0 0 0 0 0 0 0 6 375K 276K 4 39 57 > > Any objections to the patch? Can we convince you to make it an option and default to off at first? vmstat's current output format has the useful property that fault columns align nicely when they are ``small enough'', and seeing them misaligned is something that triggers a ``we have considerable load'' double-check. Right now my backup script is running, and I see: : $ vmstat -w 1 : procs memory page disks faults cpu : r b w avm fre flt re pi po fr sr ad0 md0 in sy cs us sy id : 3 0 0 1071M 211M 903 0 0 0 1176 388 0 0 661 16032 5783 12 9 79 : 3 0 0 1071M 210M 1 0 0 0 30 0 74 0 1356 3062 7927 7 15 78 : 0 0 0 1071M 209M 20 0 0 0 32 0 79 0 1509 3139 8629 4 16 80 : 0 0 0 1071M 207M 0 0 0 0 30 0 74 0 1306 2988 7653 4 14 82 : 0 0 0 1071M 206M 0 0 0 0 31 0 63 0 1315 2787 7302 6 17 77 : 1 0 0 1071M 205M 0 0 0 0 30 0 49 0 1787 2786 8750 4 20 76 : 0 0 0 1071M 204M 0 0 0 0 30 0 49 0 1974 2465 9480 5 19 76 : 0 0 0 1071M 202M 0 0 0 0 30 0 89 0 1388 3588 8432 6 16 78 : 0 0 0 1071M 200M 1 0 0 0 30 0 87 0 1358 3338 8199 6 17 77 : 3 0 0 1071M 199M 0 0 0 0 30 0 80 0 1367 4142 8216 9 16 75 : 1 0 0 1071M 198M 0 0 0 0 30 0 76 0 1339 3175 8133 6 18 76 : 1 0 0 1071M 241M 0 0 0 0 11354 0 73 0 1361 3055 7839 8 24 68 : 3 0 0 1057M 253M 0 0 0 0 3426 0 82 0 1398 3548 8273 10 17 73 : 2 0 0 1057M 247M 0 0 0 0 31 0 123 0 1357 4377 8113 4 14 82 : 1 0 0 1057M 240M 36 0 1 0 47 0 150 0 1394 4597 8526 6 16 78 : 1 0 0 1057M 233M 0 0 0 0 30 0 146 0 1298 4496 7970 4 15 81 : ^C Only the first line is misaligned by _one_ character, and I *like* the extra granularity of the rest. Your patched version has a shorter `avm' and `fre' too. Do we really have to trim them? They seem to be doing fine with physmem >= 3 GB. From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 00:19:31 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1315D106567E for ; Wed, 5 Nov 2008 00:19:31 +0000 (UTC) (envelope-from maksim.yevmenkin@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.156]) by mx1.freebsd.org (Postfix) with ESMTP id 96BE88FC1F for ; Wed, 5 Nov 2008 00:19:30 +0000 (UTC) (envelope-from maksim.yevmenkin@gmail.com) Received: by fg-out-1718.google.com with SMTP id l26so2825313fgb.35 for ; Tue, 04 Nov 2008 16:19:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type:content-transfer-encoding :content-disposition; bh=cTQC3w7XkvcA/zU9RZ+6XXSdH/MM9xx1umc7AJxHDFM=; b=ruu5PpvLoK/MCvurpr45Csfbgbv8G5+lNksaWjcrn7jfSLlo+v7Lh6Y/e0XLrVe15r IKv/B89OOoSkcH4n/T6UUtIktLTVQ5JlKeWx+FhmXtC64DeKE8DMTBdW3+spghNOpwr9 eLIWMov6mw2axB05NQ9DMT/onHWZKlO4THyig= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type :content-transfer-encoding:content-disposition; b=j+lAD1bjskMRWkoH8I6swSzs/EZyVLEUEeUaoBZ/Q36bycrf5oYfx+pEzumd4dxair N6u5H3MKToQ7jhVvQBMg4O0boXvZICn7LWGk3xTXXmV2wndNVm5O5BttFxhoklXUKbiT XlTDsU90+63/0S/pQSxZd5pF7ZZFAfXbXbiyw= Received: by 10.86.62.3 with SMTP id k3mr319319fga.46.1225842897556; Tue, 04 Nov 2008 15:54:57 -0800 (PST) Received: by 10.86.9.3 with HTTP; Tue, 4 Nov 2008 15:54:57 -0800 (PST) Message-ID: Date: Tue, 4 Nov 2008 15:54:57 -0800 From: "Maksim Yevmenkin" To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: converting strings from utf8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 00:19:31 -0000 Hello, can i use wcstombs(3) to convert a string presented in utf8 into current locale? basically i'm looking for something like iconv from ports but included into base system. in other words, would something like this work? char *locale, dst[256]; size_t len; locale = setlocale(LC_CTYPE, ""); if (locale == NULL) /* bail */ len = wcstombs(dst, (wchar_t *) src, sizeof(dst)); if (len < 0) /* bail */ printf("%*.*s\n", len, len, dst); thanks, max From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 00:49:40 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F8A31065675 for ; Wed, 5 Nov 2008 00:49:40 +0000 (UTC) (envelope-from phocking@fsklaw.com) Received: from thor-new.fsklaw.com (thor-new.fsklaw.com [64.174.116.34]) by mx1.freebsd.org (Postfix) with ESMTP id 56F7F8FC17 for ; Wed, 5 Nov 2008 00:49:40 +0000 (UTC) (envelope-from phocking@fsklaw.com) Received: from localhost (localhost [127.0.0.1]) by thor-new.fsklaw.com (Postfix) with ESMTP id 0D44321978CC; Tue, 4 Nov 2008 16:32:35 -0800 (PST) Received: from thor-new.fsklaw.com ([127.0.0.1]) by localhost (localhost [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 28000-02; Tue, 4 Nov 2008 16:32:31 -0800 (PST) Received: from [192.168.61.171] (drybook.fsklaw.com [192.168.61.171]) by thor-new.fsklaw.com (Postfix) with ESMTP id 4835621978A8; Tue, 4 Nov 2008 16:32:31 -0800 (PST) Message-ID: <4910E9A0.5000708@fsklaw.com> Date: Tue, 04 Nov 2008 16:32:32 -0800 From: Phillip Hocking User-Agent: Thunderbird 2.0.0.17 (X11/20080925) MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at fsklaw.com X-Mailman-Approved-At: Wed, 05 Nov 2008 01:01:14 +0000 Cc: zaphod@fsklaw.com Subject: Kernel panic on 7.0-RELEASE X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 00:49:40 -0000 Hello all, I keep getting a kernel panic every Saturday night, so I figured I would go through the core dump. # uname -a FreeBSD xx.fsklaw.com 7.0-RELEASE FreeBSD 7.0-RELEASE #1: Wed Apr 23 08:01:10 PDT 2008 xx@to-new.fsklaw.com:/usr/obj/usr/src/sys/TEST amd64 to# kgdb kernel.symbols /var/crash/vmcore.2 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd". Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0x250 fault code = supervisor read data, page not present instruction pointer = 0x8:0xffffffff80482a5d stack pointer = 0x10:0xffffffffb184b570 frame pointer = 0x10:0xffffff003f9cf6a0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 37920 (bsdtar) trap number = 12 panic: page fault cpuid = 1 Uptime: 7d2h23m33s Physical memory: 2034 MB Dumping 415 MB: 400 384 368 352 336 320 304 288 272 256 240 224 208 192 176 160 144 128 112 96 80 64 48 32 16 #0 doadump () at pcpu.h:194 194 __asm __volatile("movq %%gs:0,%0" : "=r" (td)); (kgdb) where #0 doadump () at pcpu.h:194 #1 0x0000000000000004 in ?? () #2 0xffffffff8048e0d9 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #3 0xffffffff8048e4dd in panic (fmt=0x104
) at /usr/src/sys/kern/kern_shutdown.c:563 #4 0xffffffff80786f44 in trap_fatal (frame=0xffffff003f9cf6a0, eva=18446742975518566608) at /usr/src/sys/amd64/amd64/trap.c:724 #5 0xffffffff80787315 in trap_pfault (frame=0xffffffffb184b4c0, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:641 #6 0xffffffff80787c58 in trap (frame=0xffffffffb184b4c0) at /usr/src/sys/amd64/amd64/trap.c:410 #7 0xffffffff8076d88e in calltrap () at /usr/src/sys/amd64/amd64/exception.S:169 #8 0xffffffff80482a5d in _mtx_lock_sleep (m=0xffffff009662c410, tid=18446742975265175200, opts=Variable "opts" is not available. ) at /usr/src/sys/kern/kern_mutex.c:335 #9 0xffffffff804f5978 in vfs_vmio_release (bp=0xffffffff9a329220) at /usr/src/sys/kern/vfs_bio.c:1530 #10 0xffffffff804f77f4 in getnewbuf (slpflag=0, slptimeo=0, size=Variable "size" is not available. ) at /usr/src/sys/kern/vfs_bio.c:1847 #11 0xffffffff804f8801 in getblk (vp=0xffffff000b71c5d0, blkno=294994, size=16384, slpflag=0, slptimeo=0, flags=Variable "flags" is not available. ) at /usr/src/sys/kern/vfs_bio.c:2602 #12 0xffffffff806a3d5b in ffs_balloc_ufs2 (vp=0xffffff000b71c5d0, startoffset=Variable "startoffset" is not available. ) at /usr/src/sys/ufs/ffs/ffs_balloc.c:844 #13 0xffffffff806bf6e5 in ffs_write (ap=0xffffffffb184ba10) at /usr/src/sys/ufs/ffs/ffs_vnops.c:720 #14 0xffffffff807e04e5 in VOP_WRITE_APV (vop=0xffffffff80a6b0e0, a=0xffffffffb184ba10) at vnode_if.c:691 #15 0xffffffff8051ae41 in vn_write (fp=0xffffff00284771e0, uio=0xffffffffb184bb00, active_cred=Variable "active_cred" is not available. ) at vnode_if.h:373 #16 0xffffffff804c6468 in dofilewrite (td=0xffffff003f9cf6a0, fd=3, fp=0xffffff00284771e0, auio=0xffffffffb184bb00, offset=Variable "offset" is not available. ) at file.h:254 #17 0xffffffff804c676e in kern_writev (td=0xffffff003f9cf6a0, fd=3, auio=0xffffffffb184bb00) at /usr/src/sys/kern/sys_generic.c:401 #18 0xffffffff804c67ec in write (td=0xffffff009662c410, uap=0xffffff003f9cf6a0) at /usr/src/sys/kern/sys_generic.c:317 #19 0xffffffff80787597 in syscall (frame=0xffffffffb184bc70) at /usr/src/sys/amd64/amd64/trap.c:852 #20 0xffffffff8076da9b in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:290 #21 0x0000000800a66d3c in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) Note the uptime, this happens every Saturday for the last three weeks. I have a cronjob that tar/bzips a drive and scps it to another box, but that runs nightly at midnight on the cronjob, and this happens weekly but only on Saturday, same cronjob same data. I figured if it happened again, I would hit all you leet-sauce developers up. Hopefully you all can suggest what to upgrade to or how to further troubleshoot this issue. Thanks a bundle y'all, love the beastie! From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 02:31:59 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ACEEB10656A3; Wed, 5 Nov 2008 02:31:58 +0000 (UTC) (envelope-from keramida@freebsd.org) Received: from igloo.linux.gr (igloo.linux.gr [62.1.205.36]) by mx1.freebsd.org (Postfix) with ESMTP id 970948FC0A; Wed, 5 Nov 2008 02:31:57 +0000 (UTC) (envelope-from keramida@freebsd.org) Received: from kobe.laptop (adsl92-78.kln.forthnet.gr [77.49.59.78]) (authenticated bits=128) by igloo.linux.gr (8.14.3/8.14.3/Debian-5) with ESMTP id mA52VdqA011613 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 5 Nov 2008 04:31:44 +0200 Received: from kobe.laptop (kobe.laptop [127.0.0.1]) by kobe.laptop (8.14.3/8.14.3) with ESMTP id mA52VdnH001416; Wed, 5 Nov 2008 04:31:39 +0200 (EET) (envelope-from keramida@freebsd.org) Received: (from keramida@localhost) by kobe.laptop (8.14.3/8.14.3/Submit) id mA52Vcm7001415; Wed, 5 Nov 2008 04:31:38 +0200 (EET) (envelope-from keramida@freebsd.org) From: Giorgos Keramidas To: Max Laier In-Reply-To: <200811042342.49827.max@love2party.net> (Max Laier's message of "Tue, 4 Nov 2008 23:42:49 +0100") Date: Wed, 05 Nov 2008 04:31:26 +0200 Message-ID: <87prlagayp.fsf@kobe.laptop> References: <200810071314.45922.max@love2party.net> <200811012114.43137.max@love2party.net> <200811042342.49827.max@love2party.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-MailScanner-ID: mA52VdqA011613 X-Hellug-MailScanner: Found to be clean X-Hellug-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-4.306, required 5, autolearn=not spam, ALL_TRUSTED -1.80, AWL 0.09, BAYES_00 -2.60) X-Hellug-MailScanner-From: keramida@freebsd.org X-Spam-Status: No Cc: freebsd-hackers@freebsd.org, koitsu@freebsd.org, pjd@freebsd.org, Pete French , lhmwzy@gmail.com Subject: Re: du -A / -B options [Re: zfs quota question] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 02:31:59 -0000 On Tue, 4 Nov 2008 23:42:49 +0100, Max Laier wrote: > Hi again, > > On Saturday 01 November 2008 21:14:42 I wrote: >> a thread on freebsd-stable@ [1] about problems with du(1) and compressed >> zfs filesystems got me looking for a possible solution. Attached is a diff >> for du(1) that adds two new options: >> >> -A to display the apparent size of the file instead of the used blocks. >> -B bsize to specify a custom blocksize. In particular one <512byte >> >> The GNU du(1) has --apparent-size for -A, but we don't like long options. >> That's not to say that it couldn't be added for script compat. -B is >> probably not that interesting, but it can be helpful and came for free. > > Attached is an updated patch. This refines the -B option to something more > useful (and fixes a bug in the original patch). > > From the man page: > > -B blocksize > Calculate block counts in blocksize byte blocks. This is differ- > ent from the -k, -m options or setting BLOCKSIZE and gives an > estimate of how many space the examined file hierachy would > require on a filesystem with the given blocksize. Unless in -A > mode, blocksize is rounded up to the next multiple of 512. That looks nice! With a small fix (``how _much_ space'') I like the idea a lot :) The patch fails to apply on a recent /head snapshot of du though: : keramida@kobe:/ws/bsd/src/usr.bin/du$ patch -p0 -i ~/du.AB.diff : [...] : The text leading up to this was: : -------------------------- : |Index: du.c : |=================================================================== : |--- du.c (revision 184656) : |+++ du.c (working copy) : -------------------------- : Patching file du.c using Plan A... : Hunk #1 failed at 79. : Hunk #2 failed at 99. : Hunk #3 failed at 159. : Hunk #4 succeeded at 215 (offset -2 lines). : Hunk #5 failed at 236. : Hunk #6 failed at 259. : Hunk #7 failed at 292. : Hunk #8 failed at 317. : Hunk #9 succeeded at 465 (offset -5 lines). : Hunk #10 succeeded at 482 (offset -2 lines). : 7 out of 10 hunks failed--saving rejects to du.c.rej : done : keramida@kobe:/ws/bsd/src/usr.bin/du$ Can you please refresh and repost it? From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 02:39:57 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F3B87106564A for ; Wed, 5 Nov 2008 02:39:56 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.186]) by mx1.freebsd.org (Postfix) with ESMTP id 6AF3F8FC1A for ; Wed, 5 Nov 2008 02:39:56 +0000 (UTC) (envelope-from max@love2party.net) Received: from vampire.homelinux.org (dslb-088-066-002-145.pools.arcor-ip.net [88.66.2.145]) by mrelayeu.kundenserver.de (node=mrelayeu3) with ESMTP (Nemesis) id 0MKxQS-1KxYJF2iI4-0007g0; Wed, 05 Nov 2008 03:39:55 +0100 Received: (qmail 25677 invoked from network); 5 Nov 2008 02:39:52 -0000 Received: from fbsd8.laiers.local (192.168.4.151) by router.laiers.local with SMTP; 5 Nov 2008 02:39:52 -0000 From: Max Laier Organization: FreeBSD To: Giorgos Keramidas Date: Wed, 5 Nov 2008 03:39:51 +0100 User-Agent: KMail/1.10.1 (FreeBSD/8.0-CURRENT; KDE/4.1.1; i386; ; ) References: <200811042342.49827.max@love2party.net> <87prlagayp.fsf@kobe.laptop> In-Reply-To: <87prlagayp.fsf@kobe.laptop> MIME-Version: 1.0 Content-Type: Multipart/Mixed; boundary="Boundary-00=_4dQEJnq7CkB11C+" Message-Id: <200811050339.52088.max@love2party.net> X-Provags-ID: V01U2FsdGVkX1+mN2+wwL1VH1rvizOT+qS4/dOp+DxTugtYMtD 2f2UYTRph79kh5EluaglgHJew0qDUxlSmgdPf9XstZ8HYXMsWn Wcv7KEGRVG8rGJaYawttQ== Cc: freebsd-hackers@freebsd.org, koitsu@freebsd.org, pjd@freebsd.org, Pete French , lhmwzy@gmail.com Subject: Re: du -A / -B options [Re: zfs quota question] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 02:39:57 -0000 --Boundary-00=_4dQEJnq7CkB11C+ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline On Wednesday 05 November 2008 03:31:26 Giorgos Keramidas wrote: > On Tue, 4 Nov 2008 23:42:49 +0100, Max Laier wrote: > > Hi again, > > > > On Saturday 01 November 2008 21:14:42 I wrote: > >> a thread on freebsd-stable@ [1] about problems with du(1) and compressed > >> zfs filesystems got me looking for a possible solution. Attached is a > >> diff for du(1) that adds two new options: > >> > >> -A to display the apparent size of the file instead of the used blocks. > >> -B bsize to specify a custom blocksize. In particular one <512byte > >> > >> The GNU du(1) has --apparent-size for -A, but we don't like long > >> options. That's not to say that it couldn't be added for script compat. > >> -B is probably not that interesting, but it can be helpful and came for > >> free. > > > > Attached is an updated patch. This refines the -B option to something > > more useful (and fixes a bug in the original patch). > > > > From the man page: > > > > -B blocksize > > Calculate block counts in blocksize byte blocks. This is > > differ- ent from the -k, -m options or setting BLOCKSIZE and gives an > > estimate of how many space the examined file hierachy would require on a > > filesystem with the given blocksize. Unless in -A mode, blocksize is > > rounded up to the next multiple of 512. > > That looks nice! With a small fix (``how _much_ space'') I like the > idea a lot :) > > The patch fails to apply on a recent /head snapshot of du though: ... > Can you please refresh and repost it? Hum ... are you sure your snapshot is up to date? I did some style(9) cleanup in r184654/6 that you might not have yet. Anyways ... here is a diff with those revisions included. -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --Boundary-00=_4dQEJnq7CkB11C+ Content-Type: text/plain; charset="iso-8859-1"; name="du.AB.earlier.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="du.AB.earlier.diff" Index: du.c =================================================================== --- du.c (revision 176561) +++ du.c (working copy) @@ -73,20 +73,21 @@ static int linkchk(FTSENT *); static void usage(void); -void prthumanval(int64_t); -void ignoreadd(const char *); -void ignoreclean(void); -int ignorep(FTSENT *); +static void prthumanval(int64_t); +static void ignoreadd(const char *); +static void ignoreclean(void); +static int ignorep(FTSENT *); -int nodumpflag = 0; +static int nodumpflag = 0; +static int Aflag; +static long blocksize, cblocksize; int main(int argc, char *argv[]) { FTS *fts; FTSENT *p; - off_t savednumber = 0; - long blocksize; + off_t savednumber, curblocks; int ftsoptions; int listall; int depth; @@ -98,79 +99,91 @@ setlocale(LC_ALL, ""); Hflag = Lflag = Pflag = aflag = sflag = dflag = cflag = hflag = - lflag = 0; + lflag = Aflag = 0; save = argv; ftsoptions = 0; + savednumber = 0; + cblocksize = DEV_BSIZE; + blocksize = 0; depth = INT_MAX; SLIST_INIT(&ignores); - while ((ch = getopt(argc, argv, "HI:LPasd:chklmnrx")) != -1) + while ((ch = getopt(argc, argv, "AB:HI:LPasd:chklmnrx")) != -1) switch (ch) { - case 'H': - Hflag = 1; - break; - case 'I': - ignoreadd(optarg); - break; - case 'L': - if (Pflag) - usage(); - Lflag = 1; - break; - case 'P': - if (Lflag) - usage(); - Pflag = 1; - break; - case 'a': - aflag = 1; - break; - case 's': - sflag = 1; - break; - case 'd': - dflag = 1; - errno = 0; - depth = atoi(optarg); - if (errno == ERANGE || depth < 0) { - warnx("invalid argument to option d: %s", optarg); - usage(); - } - break; - case 'c': - cflag = 1; - break; - case 'h': - if (setenv("BLOCKSIZE", "512", 1) == -1) - warn( - "setenv: cannot set BLOCKSIZE=512"); - hflag = 1; - break; - case 'k': - hflag = 0; - if (setenv("BLOCKSIZE", "1024", 1) == -1) - warn("setenv: cannot set BLOCKSIZE=1024"); - break; - case 'l': - lflag = 1; - break; - case 'm': - hflag = 0; - if (setenv("BLOCKSIZE", "1048576", 1) == -1) - warn("setenv: cannot set BLOCKSIZE=1048576"); - break; - case 'n': - nodumpflag = 1; - break; - case 'r': /* Compatibility. */ - break; - case 'x': - ftsoptions |= FTS_XDEV; - break; - case '?': - default: + case 'A': + Aflag = 1; + break; + case 'B': + errno = 0; + cblocksize = atoi(optarg); + if (errno == ERANGE || cblocksize < 0) { + warnx("invalid argument to option B: %s", + optarg); usage(); + } + break; + case 'H': + Hflag = 1; + break; + case 'I': + ignoreadd(optarg); + break; + case 'L': + if (Pflag) + usage(); + Lflag = 1; + break; + case 'P': + if (Lflag) + usage(); + Pflag = 1; + break; + case 'a': + aflag = 1; + break; + case 's': + sflag = 1; + break; + case 'd': + dflag = 1; + errno = 0; + depth = atoi(optarg); + if (errno == ERANGE || depth < 0) { + warnx("invalid argument to option d: %s", + optarg); + usage(); + } + break; + case 'c': + cflag = 1; + break; + case 'h': + hflag = 1; + break; + case 'k': + hflag = 0; + blocksize = 1024; + break; + case 'l': + lflag = 1; + break; + case 'm': + hflag = 0; + blocksize = 1048576; + break; + case 'n': + nodumpflag = 1; + break; + case 'r': /* Compatibility. */ + break; + case 'x': + ftsoptions |= FTS_XDEV; + break; + case '?': + default: + usage(); + /* NOTREACHED */ } argc -= optind; @@ -204,6 +217,9 @@ if (Pflag) ftsoptions |= FTS_PHYSICAL; + if (!Aflag && (cblocksize % DEV_BSIZE) != 0) + cblocksize = howmany(cblocksize, DEV_BSIZE) * DEV_BSIZE; + listall = 0; if (aflag) { @@ -222,9 +238,14 @@ argv[1] = NULL; } - (void) getbsize(¬used, &blocksize); - blocksize /= 512; + if (blocksize == 0) + (void)getbsize(¬used, &blocksize); + if (!Aflag) { + cblocksize /= DEV_BSIZE; + blocksize /= DEV_BSIZE; + } + rval = 0; if ((fts = fts_open(argv, ftsoptions, NULL)) == NULL) @@ -232,57 +253,63 @@ while ((p = fts_read(fts)) != NULL) { switch (p->fts_info) { - case FTS_D: /* Ignore. */ - if (ignorep(p)) - fts_set(fts, p, FTS_SKIP); + case FTS_D: /* Ignore. */ + if (ignorep(p)) + fts_set(fts, p, FTS_SKIP); + break; + case FTS_DP: + if (ignorep(p)) break; - case FTS_DP: - if (ignorep(p)) - break; - p->fts_parent->fts_bignum += - p->fts_bignum += p->fts_statp->st_blocks; + curblocks = Aflag ? + howmany(p->fts_statp->st_size, cblocksize) : + howmany(p->fts_statp->st_blocks, cblocksize); + p->fts_parent->fts_bignum += p->fts_bignum += + curblocks; - if (p->fts_level <= depth) { - if (hflag) { - (void) prthumanval(howmany(p->fts_bignum, blocksize)); - (void) printf("\t%s\n", p->fts_path); - } else { - (void) printf("%jd\t%s\n", - (intmax_t)howmany(p->fts_bignum, blocksize), - p->fts_path); - } + if (p->fts_level <= depth) { + if (hflag) { + prthumanval(p->fts_bignum); + (void)printf("\t%s\n", p->fts_path); + } else { + (void)printf("%jd\t%s\n", + howmany(p->fts_bignum * cblocksize, + blocksize), p->fts_path); } + } + break; + case FTS_DC: /* Ignore. */ + break; + case FTS_DNR: /* Warn, continue. */ + case FTS_ERR: + case FTS_NS: + warnx("%s: %s", p->fts_path, strerror(p->fts_errno)); + rval = 1; + break; + default: + if (ignorep(p)) break; - case FTS_DC: /* Ignore. */ + + if (lflag == 0 && p->fts_statp->st_nlink > 1 && + linkchk(p)) break; - case FTS_DNR: /* Warn, continue. */ - case FTS_ERR: - case FTS_NS: - warnx("%s: %s", p->fts_path, strerror(p->fts_errno)); - rval = 1; - break; - default: - if (ignorep(p)) - break; - if (lflag == 0 && - p->fts_statp->st_nlink > 1 && linkchk(p)) - break; + curblocks = Aflag ? + howmany(p->fts_statp->st_size, cblocksize) : + howmany(p->fts_statp->st_blocks, cblocksize); - if (listall || p->fts_level == 0) { - if (hflag) { - (void) prthumanval(howmany(p->fts_statp->st_blocks, - blocksize)); - (void) printf("\t%s\n", p->fts_path); - } else { - (void) printf("%jd\t%s\n", - (intmax_t)howmany(p->fts_statp->st_blocks, blocksize), - p->fts_path); - } + if (listall || p->fts_level == 0) { + if (hflag) { + prthumanval(curblocks); + (void)printf("\t%s\n", p->fts_path); + } else { + (void)printf("%jd\t%s\n", + howmany(curblocks * cblocksize, + blocksize), p->fts_path); } + } - p->fts_parent->fts_bignum += p->fts_statp->st_blocks; + p->fts_parent->fts_bignum += curblocks; } savednumber = p->fts_parent->fts_bignum; } @@ -292,10 +319,11 @@ if (cflag) { if (hflag) { - (void) prthumanval(howmany(savednumber, blocksize)); - (void) printf("\ttotal\n"); + prthumanval(savednumber); + (void)printf("\ttotal\n"); } else { - (void) printf("%jd\ttotal\n", (intmax_t)howmany(savednumber, blocksize)); + (void)printf("%jd\ttotal\n", (intmax_t)howmany( + savednumber * cblocksize, blocksize)); } } @@ -348,7 +376,8 @@ free_list = le->next; free(le); } - new_buckets = malloc(new_size * sizeof(new_buckets[0])); + new_buckets = malloc(new_size * + sizeof(new_buckets[0])); } if (new_buckets == NULL) { @@ -436,12 +465,14 @@ return (0); } -void +static void prthumanval(int64_t bytes) { char buf[5]; - bytes *= DEV_BSIZE; + bytes *= cblocksize; + if (!Aflag) + bytes *= DEV_BSIZE; humanize_number(buf, sizeof(buf), bytes, "", HN_AUTOSCALE, HN_B | HN_NOSPACE | HN_DECIMAL); @@ -453,12 +484,13 @@ usage(void) { (void)fprintf(stderr, - "usage: du [-H | -L | -P] [-a | -s | -d depth] [-c] " - "[-l] [-h | -k | -m] [-n] [-x] [-I mask] [file ...]\n"); + "usage: du [-A] [-H | -L | -P] [-a | -s | -d depth] [-c] " + "[-l] [-h | -k | -m | -B bsize] [-n] [-x] [-I mask] " + "[file ...]\n"); exit(EX_USAGE); } -void +static void ignoreadd(const char *mask) { struct ignentry *ign; @@ -472,7 +504,7 @@ SLIST_INSERT_HEAD(&ignores, ign, next); } -void +static void ignoreclean(void) { struct ignentry *ign; @@ -485,7 +517,7 @@ } } -int +static int ignorep(FTSENT *ent) { struct ignentry *ign; Index: du.1 =================================================================== --- du.1 (revision 184666) +++ du.1 (working copy) @@ -40,11 +40,12 @@ .Nd display disk usage statistics .Sh SYNOPSIS .Nm +.Op Fl A .Op Fl H | L | P .Op Fl a | s | d Ar depth .Op Fl c .Op Fl l -.Op Fl h | k | m +.Op Fl h | k | m | B Ar blocksize .Op Fl n .Op Fl x .Op Fl I Ar mask @@ -60,6 +61,25 @@ .Pp The options are as follows: .Bl -tag -width indent +.It Fl A +Display the apparent size instead of the disk usage. +This can be helpful when operating on compressed volumes or sparse files. +.It Fl B Ar blocksize +Calculate block counts in +.Ar blocksize +byte blocks. +This is different from the +.Fl k, m +options or setting +.Ev BLOCKSIZE +and gives an estimate of how many space the examined file hierachy would +require on a filesystem with the given +.Ar blocksize . +Unless in +.Fl A +mode, +.Ar blocksize +is rounded up to the next multiple of 512. .It Fl H Symbolic links on the command line are followed, symbolic links in file hierarchies are not followed. @@ -136,14 +156,19 @@ If the environment variable .Ev BLOCKSIZE is set, and the -.Fl k -option is not specified, the block counts will be displayed in units of that -size block. +.Fl k, m +or +.Fl h +options are not specified, the block counts will be displayed in units of +that size block. If .Ev BLOCKSIZE is not set, and the -.Fl k -option is not specified, the block counts will be displayed in 512-byte blocks. +.Fl k, m +or +.Fl h +options are not specified, the block counts will be displayed in 512-byte +blocks. .El .Sh SEE ALSO .Xr df 1 , --Boundary-00=_4dQEJnq7CkB11C+-- From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 02:55:12 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9AC141065672; Wed, 5 Nov 2008 02:55:12 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from igloo.linux.gr (igloo.linux.gr [62.1.205.36]) by mx1.freebsd.org (Postfix) with ESMTP id 136918FC18; Wed, 5 Nov 2008 02:55:11 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from kobe.laptop (adsl92-78.kln.forthnet.gr [77.49.59.78]) (authenticated bits=128) by igloo.linux.gr (8.14.3/8.14.3/Debian-5) with ESMTP id mA52t3sw012539 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 5 Nov 2008 04:55:08 +0200 Received: from kobe.laptop (kobe.laptop [127.0.0.1]) by kobe.laptop (8.14.3/8.14.3) with ESMTP id mA52t3Ze001795; Wed, 5 Nov 2008 04:55:03 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) Received: (from keramida@localhost) by kobe.laptop (8.14.3/8.14.3/Submit) id mA52t3hw001794; Wed, 5 Nov 2008 04:55:03 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) From: Giorgos Keramidas To: Max Laier In-Reply-To: <200811050339.52088.max@love2party.net> (Max Laier's message of "Wed, 5 Nov 2008 03:39:51 +0100") Date: Wed, 05 Nov 2008 04:54:56 +0200 Message-ID: <87myge6fwf.fsf@kobe.laptop> References: <200811042342.49827.max@love2party.net> <87prlagayp.fsf@kobe.laptop> <200811050339.52088.max@love2party.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-MailScanner-ID: mA52t3sw012539 X-Hellug-MailScanner: Found to be clean X-Hellug-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-3.854, required 5, autolearn=not spam, ALL_TRUSTED -1.80, AWL 0.55, BAYES_00 -2.60) X-Hellug-MailScanner-From: keramida@ceid.upatras.gr X-Spam-Status: No Cc: freebsd-hackers@freebsd.org, koitsu@freebsd.org, pjd@freebsd.org, Pete French , lhmwzy@gmail.com Subject: Re: du -A / -B options [Re: zfs quota question] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 02:55:12 -0000 On Wed, 5 Nov 2008 03:39:51 +0100, Max Laier wrote: >> That looks nice! With a small fix (``how _much_ space'') I like the >> idea a lot :) >> >> The patch fails to apply on a recent /head snapshot of du though: > ... >> Can you please refresh and repost it? > > Hum ... are you sure your snapshot is up to date? I did some style(9) > cleanup in r184654/6 that you might not have yet. Anyways ... here is > a diff with those revisions included. Ah, that's it then. I ran 'svn up' at the change right *before* the HPS-USB commit and the latest du stuff was missing. Thanks :) The patch works fine here. I only had to make a tiny change, to avoid a core dump when the -B blocksize is invalid as an integer and it caused a division by zero in fts: $ ./du -B abcde Floating point exception: 8 (core dumped) Here's a diff of all the local changes I made: %%% diff -r a3da3ebb57ab usr.bin/du/du.1 --- a/usr.bin/du/du.1 Wed Nov 05 04:47:07 2008 +0200 +++ b/usr.bin/du/du.1 Wed Nov 05 04:53:28 2008 +0200 @@ -72,7 +72,7 @@ .Fl k, m options or setting .Ev BLOCKSIZE -and gives an estimate of how many space the examined file hierachy would +and gives an estimate of how much space the examined file hierachy would require on a filesystem with the given .Ar blocksize . Unless in @@ -160,7 +160,7 @@ or .Fl h options are not specified, the block counts will be displayed in units of -that size block. +that block size. If .Ev BLOCKSIZE is not set, and the diff -r a3da3ebb57ab usr.bin/du/du.c --- a/usr.bin/du/du.c Wed Nov 05 04:47:07 2008 +0200 +++ b/usr.bin/du/du.c Wed Nov 05 04:53:28 2008 +0200 @@ -117,7 +117,7 @@ case 'B': errno = 0; cblocksize = atoi(optarg); - if (errno == ERANGE || cblocksize < 0) { + if (errno == ERANGE || cblocksize <= 0) { warnx("invalid argument to option B: %s", optarg); usage(); %%% From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 12:50:44 2008 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39CA61065670 for ; Wed, 5 Nov 2008 12:50:44 +0000 (UTC) (envelope-from steve@localhost.lu) Received: from zimbra.ion.lu (zimbra.ion.lu [80.90.47.180]) by mx1.freebsd.org (Postfix) with ESMTP id BFF088FC1B for ; Wed, 5 Nov 2008 12:50:43 +0000 (UTC) (envelope-from steve@localhost.lu) Received: from localhost (localhost [127.0.0.1]) by zimbra.ion.lu (Postfix) with ESMTP id 0260C6A4026 for ; Wed, 5 Nov 2008 13:34:27 +0100 (CET) X-Virus-Scanned: amavisd-new at X-Spam-Flag: NO X-Spam-Score: -2.431 X-Spam-Level: X-Spam-Status: No, score=-2.431 tagged_above=-10 required=6.6 tests=[AWL=0.168, BAYES_00=-2.599] Received: from zimbra.ion.lu ([127.0.0.1]) by localhost (zimbra.ion.lu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id boqU6UMklOa7 for ; Wed, 5 Nov 2008 13:34:24 +0100 (CET) Received: from [192.168.0.234] (vodsl-8662.vo.lu [85.93.200.214]) by zimbra.ion.lu (Postfix) with ESMTP id E197F6A4019 for ; Wed, 5 Nov 2008 13:34:23 +0100 (CET) Message-ID: <491192CD.4050306@localhost.lu> Date: Wed, 05 Nov 2008 13:34:21 +0100 From: Steve Clement User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: freebsd-hackers@FreeBSD.org X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: Compile issues on 7.0-p5 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 12:50:44 -0000 Hi List, I have the following altq issue: /usr/src/sys/netinet/tcp_usrreq.c: In function 'tcp_fill_info': /usr/src/sys/netinet/tcp_usrreq.c:1232: error: 'struct tcp_info' has no member named 'tcpi_rcv_nxt' it seems that tcp.h has either been updated or something else has changed. the following altq options are in my kernel: # ALTQ Support options ALTQ options ALTQ_CBQ options ALTQ_RED options ALTQ_RIO options ALTQ_HFSC options ALTQ_CDNR options ALTQ_PRIQ Cheers for help, Steve cc -c -O2 -pipe -fno-strict-aliasing -march=pentium4 -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -Werror /usr/src/sys/netinet/tcp_timer.c cc -c -O2 -pipe -fno-strict-aliasing -march=pentium4 -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -Werror /usr/src/sys/netinet/tcp_timewait.c cc -c -O2 -pipe -fno-strict-aliasing -march=pentium4 -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -mno-align-long-strings -mpreferred-stack-boundary=2 -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -ffreestanding -Werror /usr/src/sys/netinet/tcp_usrreq.c /usr/src/sys/netinet/tcp_usrreq.c: In function 'tcp_fill_info': /usr/src/sys/netinet/tcp_usrreq.c:1232: error: 'struct tcp_info' has no member named 'tcpi_rcv_nxt' /usr/src/sys/netinet/tcp_usrreq.c:1235: error: 'struct tcp_info' has no member named 'tcpi_snd_nxt' /usr/src/sys/netinet/tcp_usrreq.c:1239: error: 'TCPI_OPT_TOE' undeclared (first use in this function) /usr/src/sys/netinet/tcp_usrreq.c:1239: error: (Each undeclared identifier is reported only once /usr/src/sys/netinet/tcp_usrreq.c:1239: error: for each function it appears in.) *** Error code 1 Stop in /usr/obj/usr/src/sys/IPF. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 16:30:37 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D2EF61065673 for ; Wed, 5 Nov 2008 16:30:37 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx4.mail.ru (fallback.mail.ru [194.67.57.14]) by mx1.freebsd.org (Postfix) with ESMTP id 8EC7C8FC23 for ; Wed, 5 Nov 2008 16:30:37 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx38.mail.ru (mx38.mail.ru [194.67.23.16]) by mx4.mail.ru (mPOP.Fallback_MX) with ESMTP id 5D14A7EFA for ; Wed, 5 Nov 2008 16:40:14 +0300 (MSK) Received: from [217.25.27.27] (port=36252 helo=[217.25.27.27]) by mx38.mail.ru with asmtp id 1KxicG-0006Ys-00 for freebsd-hackers@freebsd.org; Wed, 05 Nov 2008 16:40:12 +0300 Message-ID: <4911A23B.7050104@mail.ru> Date: Wed, 05 Nov 2008 17:40:11 +0400 From: rihad User-Agent: Icedove 1.5.0.14eol (X11/20080724) MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam: Not detected X-Mras: OK X-Mailman-Approved-At: Wed, 05 Nov 2008 16:46:55 +0000 Subject: Asynchronous pipe I/O X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rihad@mail.ru List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 16:30:37 -0000 Imagine this shell pipeline: sh prog1 | sh prog2 As given above, prog1 blocks if prog2 hasn't yet read previously written data (actually, newline separated commands) or is busy. What I want is for prog1 to never block: sh prog1 | buffer | sh prog2 I first thought that the aptly named misc/buffer port would do exactly what I wanted: buffering prog1 output for prog2 to read it at its earliest convenience. That way prog1 would never block (unless it hit buffer's memory limits). Alas, misc/buffer was originally designed for tape backups, and despite its author's stating its applicability to other uses: "This is a program designed initially to speed up writing tapes on remote tape drives, but may be used as a general pipe buffering utility." buffer never starts writing unless the limit given by -s is crossed, which is 10 kbytes by default and cannot be less than 496 bytes, which is too much for me. Ideally I want it to start writing immediately, whenever new data hits its pools. Unfortunately, the -p 0 option doesn't work either: -s size Size in bytes of each block. The default blocksize is 10k to match the normal output of the tar(1) program. -p percentage Only start a write when the given percentage of the internal queue is full. A percentage around 75 often proves best. Defaults to zero. Wouldn't such an intermediary tool be a great way to boost performance for certain types of solutions? Thanks for any tips (Sorry if this was an inappropriate place to ask) From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 17:08:04 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 90615106567C for ; Wed, 5 Nov 2008 17:08:04 +0000 (UTC) (envelope-from mezz7@cox.net) Received: from eastrmmtao103.cox.net (eastrmmtao103.cox.net [68.230.240.9]) by mx1.freebsd.org (Postfix) with ESMTP id 261388FC21 for ; Wed, 5 Nov 2008 17:08:04 +0000 (UTC) (envelope-from mezz7@cox.net) Received: from eastrmimpo01.cox.net ([68.1.16.119]) by eastrmmtao103.cox.net (InterMail vM.7.08.02.01 201-2186-121-102-20070209) with ESMTP id <20081105170802.EVIC27754.eastrmmtao103.cox.net@eastrmimpo01.cox.net> for ; Wed, 5 Nov 2008 12:08:02 -0500 Received: from localhost ([68.103.35.214]) by eastrmimpo01.cox.net with bizsmtp id bV831a0024dCcn002V83t7; Wed, 05 Nov 2008 12:08:03 -0500 X-Authority-Analysis: v=1.0 c=1 a=BJYOUj_aXYIA:10 a=_ErIr5gE0BMA:10 a=e0E1ilLqAAAA:8 a=6I5d2MoRAAAA:8 a=LS5Y-Q-YuX6K_S6nM-8A:9 a=OwSUQDXHe8DwpwyDqAkjjXGe6akA:4 a=4vB-4DCPJfMA:10 a=SV7veod9ZcQA:10 a=b8hG5vVbyAkA:10 X-CM-Score: 0.00 Date: Wed, 05 Nov 2008 11:08:11 -0600 To: freebsd-hackers@freebsd.org From: "Jeremy Messenger" Content-Type: text/plain; format=flowed; delsp=yes; charset=utf-8 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Message-ID: User-Agent: Opera Mail/9.62 (Linux) Subject: preallocate files without (posix_)fallocate or fcntl's F_PREALLOCATE? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 17:08:04 -0000 Hello folks, At first, I don't really know C that well so... It's more of request or suggest if anyone is bored or interest to port on FreeBSD. :-) Recently, Transmission has added preallocate files to prevent disk fragmentation but FreeBSD does not has any of (posix_)fallocate or fcntl's F_PREALLOCATE (checked in RELENG_7's src/* and manpages). Here's what it looks like: ---------------------------------------- static int preallocateFile( int fd UNUSED, uint64_t length UNUSED ) { #ifdef HAVE_FALLOCATE return fallocate( fd, 0, offset, length ); #elif defined(HAVE_POSIX_FALLOCATE) return posix_fallocate( fd, 0, length ); #elif defined(SYS_DARWIN) fstore_t fst; fst.fst_flags = F_ALLOCATECONTIG; fst.fst_posmode = F_PEOFPOSMODE; fst.fst_offset = 0; fst.fst_length = length; fst.fst_bytesalloc = 0; return fcntl( fd, F_PREALLOCATE, &fst ); #else #warning no known method to preallocate files on this platform return -1; #endif } ---------------------------------------- You can see a bit more over at http://trac.transmissionbt.com/changeset/7051 .. You can also point me to some sources if it exists for I can try do it by myself, but with lacking C knowledge is a bit hard for me at the most of time. I don't know if UFS does not need or does need it. If anyone is interesting to write simple one for FreeBSD will be cool. Thanks! Cheers, Mezz -- mezz7@cox.net - mezz@FreeBSD.org FreeBSD GNOME Team http://www.FreeBSD.org/gnome/ - gnome@FreeBSD.org From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 17:10:27 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 494CB106564A for ; Wed, 5 Nov 2008 17:10:27 +0000 (UTC) (envelope-from neldredge@math.ucsd.edu) Received: from euclid.ucsd.edu (euclid.ucsd.edu [132.239.145.52]) by mx1.freebsd.org (Postfix) with ESMTP id 2E7AC8FC1B for ; Wed, 5 Nov 2008 17:10:26 +0000 (UTC) (envelope-from neldredge@math.ucsd.edu) Received: from zeno.ucsd.edu (zeno.ucsd.edu [132.239.145.22]) by euclid.ucsd.edu (8.11.7p3+Sun/8.11.7) with ESMTP id mA5HAQ614477; Wed, 5 Nov 2008 09:10:26 -0800 (PST) Received: from localhost (neldredg@localhost) by zeno.ucsd.edu (8.11.7p3+Sun/8.11.7) with ESMTP id mA5HAQk11810; Wed, 5 Nov 2008 09:10:26 -0800 (PST) X-Authentication-Warning: zeno.ucsd.edu: neldredg owned process doing -bs Date: Wed, 5 Nov 2008 09:10:26 -0800 (PST) From: Nate Eldredge X-X-Sender: neldredg@zeno.ucsd.edu To: rihad In-Reply-To: <4911A23B.7050104@mail.ru> Message-ID: References: <4911A23B.7050104@mail.ru> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-hackers@freebsd.org Subject: Re: Asynchronous pipe I/O X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 17:10:27 -0000 On Wed, 5 Nov 2008, rihad wrote: > Imagine this shell pipeline: > > sh prog1 | sh prog2 > > > As given above, prog1 blocks if prog2 hasn't yet read previously written > data (actually, newline separated commands) or is busy. What I want is > for prog1 to never block: > > sh prog1 | buffer | sh prog2 [and misc/buffer is unsuitable] I found an old piece of code laying around that I wrote for this purpose. Looking at it, I can see a number of inefficiencies, but it might do in a pinch. You're welcome to use it; I hereby release it to the public domain. Another hack that you could use, if you don't mind storing the buffer on disk rather than memory, is sh prog1 > tmpfile & tail -f -c +0 tmpfile | sh prog2 Here's my program. /* Buffering filter. */ #include #include #include #include #include #include /* Size of a single buffer. */ #define BUFSIZE 512 struct buffer { struct buffer *next; size_t length; unsigned char buf[BUFSIZE]; }; struct buffer *reader; struct buffer *writer; int max_mem = 100 * 1024; int current_mem; #define OK 0 #define WAIT 1 #define GIVEUP 2 int read_one (int fd) { int result; if (current_mem > (max_mem - sizeof(*reader->next))) { fprintf(stderr, "Reached max_mem!\n"); return WAIT; } /* Get a new buffer. */ reader->next = malloc(sizeof(*reader->next)); if (reader->next) { current_mem += sizeof(*reader->next); fprintf(stderr, "\rReading: \t%u bytes in buffer ", current_mem); } else { fprintf(stderr, "Virtual memory exhausted\n"); return WAIT; } reader = reader->next; reader->next = NULL; result = read(fd, reader->buf, BUFSIZE); if (result > 0) reader->length = result; else if (result == 0) { fprintf(stderr, "Hit EOF on reader\n"); return GIVEUP; } else if (result < 0) { fprintf(stderr, "Error on reader: %s\n", strerror(errno)); return GIVEUP; } return OK; } int write_one (int fd) { struct buffer *newwriter; if (reader == writer) return WAIT; /* the reader owns the last buffer */ if (writer->length > 0) { int result; result = write(fd, writer->buf, writer->length); if (result == 0) { fprintf(stderr, "Hit EOF on writer\n"); return GIVEUP; } else if (result < 0) { fprintf(stderr, "Error on writer: %s\n", strerror(errno)); return GIVEUP; } } newwriter = writer->next; free(writer); current_mem -= sizeof(*writer); fprintf(stderr, "\rWriting: \t%u bytes in buffer ", current_mem); writer = newwriter; return OK; } void move_data(int in_fd, int out_fd) { int reader_state = OK; int writer_state = OK; int maxfd = ((in_fd > out_fd) ? in_fd : out_fd) + 1; reader = malloc(sizeof(*reader)); if (!reader) { fprintf(stderr, "No memory at all!\n"); return; } reader->next = NULL; reader->length = 0; writer = reader; current_mem = sizeof(*reader); while (1) /* break when done */ { int result; fd_set read_set, write_set; FD_ZERO(&read_set); FD_ZERO(&write_set); if (reader_state == OK) FD_SET(in_fd, &read_set); if (writer_state == OK) FD_SET(out_fd, &write_set); result = select(maxfd, &read_set, &write_set, NULL, NULL); /* If we're ready to do something, do it. Also let the other end get a chance if something changed. */ if (FD_ISSET(in_fd, &read_set)) { reader_state = read_one(in_fd); if (writer_state == WAIT) writer_state = OK; } if (FD_ISSET(out_fd, &write_set)) { writer_state = write_one(out_fd); if (reader_state == WAIT) reader_state = OK; } /* Check for termination */ if (writer_state == GIVEUP) break; /* can't write any more */ if (reader_state == GIVEUP && writer_state == WAIT) break; /* can't read any more, and wrote all we have */ } } int main(void) { move_data(0, 1); return 0; } -- Nate Eldredge neldredge@math.ucsd.edu From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 18:13:12 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E7B75106567A for ; Wed, 5 Nov 2008 18:13:12 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from mail11.syd.optusnet.com.au (mail11.syd.optusnet.com.au [211.29.132.192]) by mx1.freebsd.org (Postfix) with ESMTP id 7CC5D8FC16 for ; Wed, 5 Nov 2008 18:13:12 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from server.vk2pj.dyndns.org (c122-106-215-175.belrs3.nsw.optusnet.com.au [122.106.215.175]) by mail11.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id mA5IDAbd029339 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 6 Nov 2008 05:13:10 +1100 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.3/8.14.3) with ESMTP id mA5ID9d7051388; Thu, 6 Nov 2008 05:13:09 +1100 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.3/8.14.3/Submit) id mA5ID9Pt051387; Thu, 6 Nov 2008 05:13:09 +1100 (EST) (envelope-from peter) Date: Thu, 6 Nov 2008 05:13:09 +1100 From: Peter Jeremy To: rihad Message-ID: <20081105181309.GB51239@server.vk2pj.dyndns.org> References: <4911A23B.7050104@mail.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="H1spWtNR+x+ondvy" Content-Disposition: inline In-Reply-To: <4911A23B.7050104@mail.ru> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-hackers@freebsd.org Subject: Re: Asynchronous pipe I/O X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 18:13:13 -0000 --H1spWtNR+x+ondvy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2008-Nov-05 17:40:11 +0400, rihad wrote: >Imagine this shell pipeline: > >sh prog1 | sh prog2 > > >As given above, prog1 blocks if prog2 hasn't yet read previously written >data (actually, newline separated commands) or is busy. What I want is >for prog1 to never block: > >sh prog1 | buffer | sh prog2 There's also misc/mbuffer which is supposed to be an enhancement of misc/buffer - though I haven't used either. I have a program I wrote to do this but it's not in a releasable state. >Wouldn't such an intermediary tool be a great way to boost performance >for certain types of solutions? I've found that for dump|restore or dump|gzip, I can get quite significant speedups by adding a buffer that is several hundred MB in the middle. --=20 Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour. --H1spWtNR+x+ondvy Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkR4jUACgkQ/opHv/APuId41ACfagHvmVdaJpqFJgbehUiudfTb HewAn1OqR5EoTgFpiJnaoTWJqvz+pkbR =5DA6 -----END PGP SIGNATURE----- --H1spWtNR+x+ondvy-- From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 18:28:04 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E171A106567F for ; Wed, 5 Nov 2008 18:28:04 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: from palm.hoeg.nl (unknown [IPv6:2001:7b8:613:100::211]) by mx1.freebsd.org (Postfix) with ESMTP id 7E2798FC1A for ; Wed, 5 Nov 2008 18:28:04 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: by palm.hoeg.nl (Postfix, from userid 1000) id 60DC31CC62; Wed, 5 Nov 2008 19:28:03 +0100 (CET) Date: Wed, 5 Nov 2008 19:28:03 +0100 From: Ed Schouten To: Jeremy Messenger Message-ID: <20081105182803.GG1165@hoeg.nl> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6nIn9DQvaie41cmo" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.18 (2008-05-17) Cc: FreeBSD Hackers Subject: Re: preallocate files without (posix_)fallocate or fcntl's F_PREALLOCATE? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 18:28:05 -0000 --6nIn9DQvaie41cmo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, Not entirely related to this topic, but looking at the standards, it seems we don't implement posix_fadvise() either. It seems we could implement this as a no-op for now. It has no real advantage if we do, but say, one day we gain real posix_fadvise() support, we already have a bunch of binaries that have these performance optimisations compiled in. Any comments? --=20 Ed Schouten WWW: http://80386.nl/ --6nIn9DQvaie41cmo Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) iEYEARECAAYFAkkR5bMACgkQ52SDGA2eCwWuKACdEF+wxaoSlo/9s+aLEj5xygl4 T2QAnRF1Yeab4JjnqnWCLQkh0Cj7qOeV =SDZZ -----END PGP SIGNATURE----- --6nIn9DQvaie41cmo-- From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 19:10:40 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3FBEC1065678 for ; Wed, 5 Nov 2008 19:10:40 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx33.mail.ru (mx33.mail.ru [194.67.23.194]) by mx1.freebsd.org (Postfix) with ESMTP id EECA48FC0C for ; Wed, 5 Nov 2008 19:10:39 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from [217.25.27.27] (port=18691 helo=[217.25.27.27]) by mx33.mail.ru with asmtp id 1Kxnm2-000F7y-00; Wed, 05 Nov 2008 22:10:38 +0300 Message-ID: <4911EFAC.7090909@mail.ru> Date: Wed, 05 Nov 2008 23:10:36 +0400 From: rihad User-Agent: Icedove 1.5.0.14eol (X11/20080724) MIME-Version: 1.0 To: Nate Eldredge References: <4911A23B.7050104@mail.ru> In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Spam: Not detected X-Mailman-Approved-At: Wed, 05 Nov 2008 19:16:33 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: Asynchronous pipe I/O X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 19:10:40 -0000 Nate Eldredge wrote: > On Wed, 5 Nov 2008, rihad wrote: > >> Imagine this shell pipeline: >> >> sh prog1 | sh prog2 >> >> >> As given above, prog1 blocks if prog2 hasn't yet read previously written >> data (actually, newline separated commands) or is busy. What I want is >> for prog1 to never block: >> >> sh prog1 | buffer | sh prog2 > > [and misc/buffer is unsuitable] > > I found an old piece of code laying around that I wrote for this > purpose. Looking at it, I can see a number of inefficiencies, but it > might do in a pinch. You're welcome to use it; I hereby release it to > the public domain. > > Another hack that you could use, if you don't mind storing the buffer on > disk rather than memory, is > > sh prog1 > tmpfile & > tail -f -c +0 tmpfile | sh prog2 > Thanks, but I was looking for how to get some port do that useful functionality for me. Strange as it is, there doesn't seem to be such a tool handy, despite the "small tool doing one thing good" philosophy of Unix! It's invaluable for optimizing certain tasks and isn't coupled to anything: as generic as could be. Perhaps I should also email buffer's author for his opinion, and ask him why -p 0 (default) doesn't do the Right Thing. > Here's my program. Thanks again, I'm sure I'll use your code if I can't get the port version to work. From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 19:21:18 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D2941065676 for ; Wed, 5 Nov 2008 19:21:18 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx27.mail.ru (mx27.mail.ru [194.67.23.23]) by mx1.freebsd.org (Postfix) with ESMTP id BC2B08FC12 for ; Wed, 5 Nov 2008 19:21:17 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from [217.25.27.27] (port=56447 helo=[217.25.27.27]) by mx27.mail.ru with asmtp id 1KxnwJ-000G2v-00; Wed, 05 Nov 2008 22:21:15 +0300 Message-ID: <4911F229.2010502@mail.ru> Date: Wed, 05 Nov 2008 23:21:13 +0400 From: rihad User-Agent: Icedove 1.5.0.14eol (X11/20080724) MIME-Version: 1.0 To: Peter Jeremy References: <4911A23B.7050104@mail.ru> <20081105181309.GB51239@server.vk2pj.dyndns.org> In-Reply-To: <20081105181309.GB51239@server.vk2pj.dyndns.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam: Not detected X-Mailman-Approved-At: Wed, 05 Nov 2008 21:22:23 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: Asynchronous pipe I/O X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 19:21:18 -0000 Peter Jeremy wrote: > On 2008-Nov-05 17:40:11 +0400, rihad wrote: >> Imagine this shell pipeline: >> >> sh prog1 | sh prog2 >> >> >> As given above, prog1 blocks if prog2 hasn't yet read previously written >> data (actually, newline separated commands) or is busy. What I want is >> for prog1 to never block: >> >> sh prog1 | buffer | sh prog2 > > There's also misc/mbuffer which is supposed to be an enhancement of > misc/buffer - though I haven't used either. I have a program I wrote > to do this but it's not in a releasable state. > That thing only works for certain types of devices (tape?): $ while :; do echo hi; sleep 1; done | mbuffer -q | cat warning: Could not stat output device (unsupported by system)! This can result in incorrect written data when using multiple volumes. Continue at your own risk! ^C $ While writing this email I tried another incantation, and buffer finally worked! $ while :; do echo hi; sleep 1; done | buffer -s1 -b100 | cat hi hi hi hi hi ^C (each line was output one second apart). Once again thank to everybody participating! >> Wouldn't such an intermediary tool be a great way to boost performance >> for certain types of solutions? > > I've found that for dump|restore or dump|gzip, I can get quite significant > speedups by adding a buffer that is several hundred MB in the middle. > Well, if OS buffers aren't good enough, then throwing some memory at disk I/O surely helps ;) From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 21:37:15 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A3D7106567E for ; Wed, 5 Nov 2008 21:37:15 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from kientzle.com (kientzle.com [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id 1F0CC8FC12 for ; Wed, 5 Nov 2008 21:37:15 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from [10.123.2.205] (p53.kientzle.com [66.166.149.53]) by kientzle.com (8.12.9/8.12.9) with ESMTP id mA5LbEtv035181; Wed, 5 Nov 2008 13:37:14 -0800 (PST) (envelope-from kientzle@freebsd.org) Message-ID: <49121206.9090804@freebsd.org> Date: Wed, 05 Nov 2008 13:37:10 -0800 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20060422 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Maksim Yevmenkin References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: converting strings from utf8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 21:37:15 -0000 Maksim Yevmenkin wrote: > > can i use wcstombs(3) to convert a string presented in utf8 into > current locale? basically i'm looking for something like iconv from > ports but included into base system. This isn't as easy as it should be, unfortunately. First, UTF-8 is itself a multibyte encoding, so you have to first convert to wide characters before you can use wcstombs(). You could in theory use the following: * Set locale to UTF-8 * use mbstowcs() to convert UTF-8 into wide characters * Set locale to your preferred locale * use wcstombs() to convert wide characters to your locale Besides being ugly, the locale names themselves are not standardized, so it's hard to do this portably. For a lot of applications, the error handling in wcstombs() is also troublesome; it rejects the entire string if any one character can't be converted. When I had to do this for libarchive, where the code had to be very portable (which precluded using iconv), I ended up doing the following: * Wrote my own converter from UTF-8 to wide characters (fortunately, UTF-8 is pretty simple to decode; this is about 20-30 lines of C) * Used wctomb() to convert one character at a time from wide characters to the current locale. I've found that wctomb() is more portable than a lot of the other functions (I think it's in C89, whereas a lot of the other standard conversion routines were introduced in C99) and provides better error-handling capabilities since it operates on one character at a time (so you can, for instance, convert characters that aren't supported in the current locale into '?' or some kind of \-escape). Feel free to copy any of my code from libarchive if it helps. Tim From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 22:51:23 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6112106567C for ; Wed, 5 Nov 2008 22:51:23 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 4135C8FC19 for ; Wed, 5 Nov 2008 22:51:23 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [IPv6:::1]) (authenticated bits=0) by server.baldwin.cx (8.14.3/8.14.3) with ESMTP id mA5MpEQp015990; Wed, 5 Nov 2008 17:51:15 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Wed, 5 Nov 2008 17:17:34 -0500 User-Agent: KMail/1.9.7 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200811051717.34933.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [IPv6:::1]); Wed, 05 Nov 2008 17:51:15 -0500 (EST) X-Virus-Scanned: ClamAV 0.93.1/8576/Wed Nov 5 13:28:48 2008 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00,NO_RELAYS autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: Jeremy Messenger Subject: Re: preallocate files without (posix_)fallocate or fcntl's F_PREALLOCATE? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 22:51:23 -0000 On Wednesday 05 November 2008 12:08:11 pm Jeremy Messenger wrote: > Hello folks, > > At first, I don't really know C that well so... It's more of request or > suggest if anyone is bored or interest to port on FreeBSD. :-) > > Recently, Transmission has added preallocate files to prevent disk > fragmentation but FreeBSD does not has any of (posix_)fallocate or fcntl's > F_PREALLOCATE (checked in RELENG_7's src/* and manpages). Here's what it > looks like: > > ---------------------------------------- > static int > preallocateFile( int fd UNUSED, uint64_t length UNUSED ) > { > #ifdef HAVE_FALLOCATE > > return fallocate( fd, 0, offset, length ); > > #elif defined(HAVE_POSIX_FALLOCATE) > > return posix_fallocate( fd, 0, length ); > > #elif defined(SYS_DARWIN) > > fstore_t fst; > fst.fst_flags = F_ALLOCATECONTIG; > fst.fst_posmode = F_PEOFPOSMODE; > fst.fst_offset = 0; > fst.fst_length = length; > fst.fst_bytesalloc = 0; > return fcntl( fd, F_PREALLOCATE, &fst ); > > #else > > #warning no known method to preallocate files on this platform > return -1; > > #endif > } > ---------------------------------------- > > You can see a bit more over at > http://trac.transmissionbt.com/changeset/7051 .. You can also point me to > some sources if it exists for I can try do it by myself, but with lacking > C knowledge is a bit hard for me at the most of time. I don't know if UFS > does not need or does need it. If anyone is interesting to write simple > one for FreeBSD will be cool. Thanks! UFS would benefit. See the notes on MAP_NOSYNC in mmap(2). Peter Wemm (peter@) probably can give a more detailed suggestion of what you would need to do. Doing it efficiently would probably require a new VOP_ALLOCATE() or some such. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 6 07:52:53 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B4F4106564A for ; Thu, 6 Nov 2008 07:52:53 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx71.mail.ru (mx71.mail.ru [194.67.23.4]) by mx1.freebsd.org (Postfix) with ESMTP id 38A838FC1B for ; Thu, 6 Nov 2008 07:52:52 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from [217.25.27.27] (port=12520 helo=[217.25.27.27]) by mx71.mail.ru with asmtp id 1Kxzff-000JZ2-00; Thu, 06 Nov 2008 10:52:51 +0300 Message-ID: <4912A251.8010604@mail.ru> Date: Thu, 06 Nov 2008 11:52:49 +0400 From: rihad User-Agent: Icedove 1.5.0.14eol (X11/20080724) MIME-Version: 1.0 To: Nate Eldredge References: <4911A23B.7050104@mail.ru> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam: Not detected X-Mras: OK X-Mailman-Approved-At: Thu, 06 Nov 2008 12:12:45 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: Asynchronous pipe I/O X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Nov 2008 07:52:53 -0000 Nate Eldredge wrote: > sh prog1 > tmpfile & > tail -f -c +0 tmpfile | sh prog2 > BTW, I don't think this would solve my problem as tail -f would block waiting for more data, as would prog2, so after prog1 finishes writing data they would block indefinitely on input. I forgot to mention this was supposed to be a short-lived pipeline, not daemons, as you had probably thought. Also, I've already learned how to use misc/buffer correctly so the problem's gone. Thank you for your time. From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 6 12:58:02 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7FC2106564A; Thu, 6 Nov 2008 12:58:02 +0000 (UTC) (envelope-from petefrench@ticketswitch.com) Received: from constantine.ticketswitch.com (constantine.ticketswitch.com [IPv6:2002:57e0:1d4e:1::3]) by mx1.freebsd.org (Postfix) with ESMTP id 8F24D8FC1F; Thu, 6 Nov 2008 12:58:02 +0000 (UTC) (envelope-from petefrench@ticketswitch.com) Received: from dilbert.rattatosk ([10.64.50.6] helo=dilbert.ticketswitch.com) by constantine.ticketswitch.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Ky4Qu-0008Qo-TB; Thu, 06 Nov 2008 12:57:56 +0000 Received: from petefrench by dilbert.ticketswitch.com with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Ky4Qp-000Cbx-IB; Thu, 06 Nov 2008 12:57:53 +0000 To: freebsd-hackers@freebsd.org, max@love2party.net In-Reply-To: <200811012114.43137.max@love2party.net> Message-Id: From: Pete French Date: Thu, 06 Nov 2008 12:57:51 +0000 X-Mailman-Approved-At: Thu, 06 Nov 2008 13:13:16 +0000 Cc: pjd@freebsd.org, koitsu@freebsd.org, lhmwzy@gmail.com Subject: Re: du -A / -B options [Re: zfs quota question] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Nov 2008 12:58:02 -0000 > Any objections against the general concept? It's rather complicated to get > the apparent size of a directory hierarchy without scripting. I often wonder > if some hierarchy will fit on a CD/DVD and compressed zfs makes this really > difficult. It's a definite thumbs up from me - this is exactly what I was looking for in the original thread. Very nice couple of new options. -pete. From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 7 03:34:58 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BAA1510656B8 for ; Fri, 7 Nov 2008 03:34:58 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from igloo.linux.gr (igloo.linux.gr [62.1.205.36]) by mx1.freebsd.org (Postfix) with ESMTP id 19CDA8FC3C for ; Fri, 7 Nov 2008 03:34:57 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from kobe.laptop (adsl202-73.kln.forthnet.gr [79.103.15.73]) (authenticated bits=128) by igloo.linux.gr (8.14.3/8.14.3/Debian-5) with ESMTP id mA73YkmK001465 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 7 Nov 2008 05:34:52 +0200 Received: from kobe.laptop (kobe.laptop [127.0.0.1]) by kobe.laptop (8.14.3/8.14.3) with ESMTP id mA73YkFF002516; Fri, 7 Nov 2008 05:34:46 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) Received: (from keramida@localhost) by kobe.laptop (8.14.3/8.14.3/Submit) id mA73YiAH002515; Fri, 7 Nov 2008 05:34:44 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) From: Giorgos Keramidas To: stevefranks@ieee.org References: <539c60b90810311123w2aa94b8akcd0a5d0fe791885a@mail.gmail.com> <539c60b90810311230i11460966la7ff35b0093642ec@mail.gmail.com> <87r65vl9ur.fsf@kobe.laptop> <539c60b90811031412y35e2599cneca5912ada43b081@mail.gmail.com> Date: Fri, 07 Nov 2008 05:34:44 +0200 In-Reply-To: <539c60b90811031412y35e2599cneca5912ada43b081@mail.gmail.com> (Steve Franks's message of "Mon, 3 Nov 2008 15:12:34 -0700") Message-ID: <87tzakqkdn.fsf@kobe.laptop> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-MailScanner-ID: mA73YkmK001465 X-Hellug-MailScanner: Found to be clean X-Hellug-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-3.133, required 5, ALL_TRUSTED -1.80, AWL -0.69, BAYES_00 -2.60, URIBL_BLACK 1.96) X-Hellug-MailScanner-From: keramida@ceid.upatras.gr X-Spam-Status: No Cc: freebsd-hackers@freebsd.org Subject: Re: includes, configure, /usr/lib vs. /usr/local/lib, and linux coders X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Nov 2008 03:34:58 -0000 On Mon, 3 Nov 2008 15:12:34 -0700, "Steve Franks" wrote: > That's alot of good info. It should go in the porter's handbook, > maybe... Hi Steve, Probably not. What I wrote is specific to the GNU build system. We have many ports that use configure scripts and makefiles generated from various versions of the GNU build tools, but ports are ``different''. We are not the _maintainers_ of the main source code of all the ported applications. We just have to make sure they build on FreeBSD, and that's pretty much all of it. For example, if the source tree of a port includes a `configure.in' that is broken on FreeBSD and Cygwin, we don't really have to ``fix'' both of these. If it builds correctly on FreeBSD, we are done. This may not be enough for Cygwin users, but we are not out to fix everyone's code to build on everybody else's system. That would be an insane amount of work for a very doubtful amount of gain :) > So, if I'm looking to make a port, which one of those people should I > be acting as? Maintainer? That's FreeBSD-port-terminology you are > using, correct? FreeBSD Porters are a separate category. They usually fall in the category of `builder' I mentioned in the original post, but they have to provide the tools for `packagers' too, in the form of Makefiles, scripts and packaging lists that allow others to configure, build and package the ``vendor'' code for some FreeBSD version. When I mentioned `maintainer', `builder' and `packager' roles in the original post I didn't mean *FreeBSD-maintainer* but the actual person or team that maintains the upstream source of a program. HTH, Giorgos From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 7 19:27:49 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5D90106564A for ; Fri, 7 Nov 2008 19:27:49 +0000 (UTC) (envelope-from sem@FreeBSD.org) Received: from mail.ciam.ru (mail.ciam.ru [212.34.63.72]) by mx1.freebsd.org (Postfix) with ESMTP id 953C08FC08 for ; Fri, 7 Nov 2008 19:27:49 +0000 (UTC) (envelope-from sem@FreeBSD.org) Received: from dhcp250-210.yandex.ru ([87.250.250.210]) by mail.ciam.ru with esmtpa (Exim 4.x) id 1KyWgK-000NcD-1z for freebsd-hackers@freebsd.org; Fri, 07 Nov 2008 22:07:44 +0300 Message-ID: <491491E5.8090805@FreeBSD.org> Date: Fri, 07 Nov 2008 22:07:17 +0300 From: Sergey Matveychuk User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: LIST_REMOVE problem X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Nov 2008 19:27:49 -0000 Hi! I wonder how this example from queue(3) works: while (!LIST_EMPTY(&head)) { /* List Deletion. */ n1 = LIST_FIRST(&head); LIST_REMOVE(n1, entries); free(n1); } If we'll take a look at code, LIST_REMOVE don't change a head pointer if you remove the first element: #define LIST_FIRST(head) ((head)->lh_first) ... #define LIST_REMOVE(elm, field) do { \ if (LIST_NEXT((elm), field) != NULL) \ LIST_NEXT((elm), field)->field.le_prev = \ (elm)->field.le_prev; \ *(elm)->field.le_prev = LIST_NEXT((elm), field); \ } while (0) So n1 in the example above will point to removed (and freed) element. -- Dixi. Sem. From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 7 19:29:54 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D4721065673 for ; Fri, 7 Nov 2008 19:29:54 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from www.sonnenberger.org (www.ostsee-abc.de [62.206.222.50]) by mx1.freebsd.org (Postfix) with ESMTP id 2CE0C8FC21 for ; Fri, 7 Nov 2008 19:29:53 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from britannica.bec.de (localhost [IPv6:::1]) by www.sonnenberger.org (Postfix) with ESMTP id DC6CD6677B for ; Fri, 7 Nov 2008 20:29:23 +0100 (CET) Received: by britannica.bec.de (Postfix, from userid 1000) id 83D49933CB; Fri, 7 Nov 2008 20:25:20 +0100 (CET) Date: Fri, 7 Nov 2008 20:25:20 +0100 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org Message-ID: <20081107192520.GA17933@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org References: <491491E5.8090805@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <491491E5.8090805@FreeBSD.org> User-Agent: Mutt/1.5.18 (2008-05-17) Subject: Re: LIST_REMOVE problem X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Nov 2008 19:29:54 -0000 On Fri, Nov 07, 2008 at 10:07:17PM +0300, Sergey Matveychuk wrote: > If we'll take a look at code, LIST_REMOVE don't change a head pointer if > you remove the first element: It does via le_priv. Joerg From owner-freebsd-hackers@FreeBSD.ORG Sat Nov 8 11:07:59 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67E471065672 for ; Sat, 8 Nov 2008 11:07:59 +0000 (UTC) (envelope-from sem@FreeBSD.org) Received: from mail.ciam.ru (mail.ciam.ru [212.34.63.72]) by mx1.freebsd.org (Postfix) with ESMTP id 276B48FC08 for ; Sat, 8 Nov 2008 11:07:59 +0000 (UTC) (envelope-from sem@FreeBSD.org) Received: from [77.41.76.79] (helo=[172.16.100.4]) by mail.ciam.ru with esmtpa (Exim 4.x) id 1Kylfa-000Lx3-4U for freebsd-hackers@freebsd.org; Sat, 08 Nov 2008 14:07:58 +0300 Message-ID: <4915730C.3050303@FreeBSD.org> Date: Sat, 08 Nov 2008 14:07:56 +0300 From: Sergey Matveychuk User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <491491E5.8090805@FreeBSD.org> <20081107192520.GA17933@britannica.bec.de> In-Reply-To: <20081107192520.GA17933@britannica.bec.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: LIST_REMOVE problem X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Nov 2008 11:07:59 -0000 Joerg Sonnenberger wrote: > On Fri, Nov 07, 2008 at 10:07:17PM +0300, Sergey Matveychuk wrote: >> If we'll take a look at code, LIST_REMOVE don't change a head pointer if >> you remove the first element: > > It does via le_priv. OK, thanks. I see now. I'll try to find out why my code don't work this way on 6.4. -- Dixi. Sem.