From owner-freebsd-fs@FreeBSD.ORG Mon Jul 24 14:33:53 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1CBF116A4E1; Mon, 24 Jul 2006 14:33:53 +0000 (UTC) (envelope-from bkoenig@cs.tu-berlin.de) Received: from efacilitas.de (smtp.efacilitas.de [85.10.196.108]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0217D43D46; Mon, 24 Jul 2006 14:32:52 +0000 (GMT) (envelope-from bkoenig@cs.tu-berlin.de) Received: from eurystheus.local (port-212-202-39-242.dynamic.qsc.de [212.202.39.242]) by efacilitas.de (Postfix) with ESMTP id 108344CCB5; Mon, 24 Jul 2006 16:32:50 +0200 (CEST) Received: from [192.168.1.2] (muhkuh.local [192.168.1.2]) by eurystheus.local (Postfix) with ESMTP id 1AA735285B; Mon, 24 Jul 2006 16:32:48 +0200 (CEST) Message-ID: <44C4DA1A.80406@cs.tu-berlin.de> Date: Mon, 24 Jul 2006 16:32:58 +0200 From: =?ISO-8859-15?Q?Bj=F6rn_K=F6nig?= User-Agent: Mozilla Thunderbird 1.0.7 (Windows/20050923) X-Accept-Language: de-DE, de, en-us, en MIME-Version: 1.0 To: etalk etalk References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org, freebsd-performance@freebsd.org Subject: Re: about the performance comparsion of 6.1 vs 5.3 on amd64 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jul 2006 14:33:53 -0000 Hello, I used unixbench among some other tools like ubench, NetPIPE, mysqlbench to get an overall impression of the performance. These are some the results of unixbench: FreeBSD 5.4-RELEASE: http://alpha-tierchen.de/dateien/etc/unixbench-amd64 FreeBSD 6.0-BETA2: http://alpha-tierchen.de/dateien/etc/unixbench-amd64-6-aj-sched_4bsd This is an AMD Athlon64 3000+ processor. Regards Björn From owner-freebsd-fs@FreeBSD.ORG Mon Jul 24 14:55:40 2006 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E0EF416A4E0 for ; Mon, 24 Jul 2006 14:55:40 +0000 (UTC) (envelope-from rick@snowhite.cis.uoguelph.ca) Received: from mailhub.cs.uoguelph.ca (mailhub.cs.uoguelph.ca [131.104.94.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id 84CC543D67 for ; Mon, 24 Jul 2006 14:55:38 +0000 (GMT) (envelope-from rick@snowhite.cis.uoguelph.ca) Received: from snowhite.cis.uoguelph.ca (snowhite.cis.uoguelph.ca [131.104.48.1]) by mailhub.cs.uoguelph.ca (8.13.1/8.13.1) with ESMTP id k6OEtagn011425 for ; Mon, 24 Jul 2006 10:55:37 -0400 Received: (from rick@localhost) by snowhite.cis.uoguelph.ca (8.9.3/8.9.3) id KAA63411 for fs@freebsd.org; Mon, 24 Jul 2006 10:56:50 -0400 (EDT) Date: Mon, 24 Jul 2006 10:56:50 -0400 (EDT) From: rick@snowhite.cis.uoguelph.ca Message-Id: <200607241456.KAA63411@snowhite.cis.uoguelph.ca> To: fs@freebsd.org X-Scanned-By: MIMEDefang 2.52 on 131.104.94.205 Cc: Subject: freebsd4.11 patch for nfs over tcp X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jul 2006 14:55:41 -0000 I figured that putting patches for FreeBSD4 in the bugtracker was passe, so I'm posting it here. I noticed that performance of nfs over TCP was abismal under certain circumstances. It turned out that TCP_NODELAY wasn't being set, because sopt_dir wasn't filled in. Here's a simple diff -c patch, rick ps: sopt_dir also isn't set for KEEPALIVE, but it doesn't matter for that one. --- FreeBSD4.11 diff -c patch *** nfs_socket.c.orig Mon Jul 24 10:37:45 2006 --- nfs_socket.c Mon Jul 24 10:38:39 2006 *************** *** 339,344 **** --- 339,345 ---- bzero(&sopt, sizeof sopt); sopt.sopt_level = IPPROTO_TCP; + sopt.sopt_dir = SOPT_SET; sopt.sopt_name = TCP_NODELAY; sopt.sopt_val = &val; sopt.sopt_valsize = sizeof val; *** nfs_syscalls.c.orig Mon Jul 24 10:37:57 2006 --- nfs_syscalls.c Mon Jul 24 10:39:12 2006 *************** *** 376,381 **** --- 376,382 ---- bzero(&sopt, sizeof sopt); sopt.sopt_level = IPPROTO_TCP; + sopt.sopt_dir = SOPT_SET; sopt.sopt_name = TCP_NODELAY; sopt.sopt_val = &val; sopt.sopt_valsize = sizeof val; From owner-freebsd-fs@FreeBSD.ORG Mon Jul 24 15:28:24 2006 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8738E16A4DA for ; Mon, 24 Jul 2006 15:28:24 +0000 (UTC) (envelope-from rees@citi.umich.edu) Received: from citi.umich.edu (citi.umich.edu [141.211.133.111]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5443843D45 for ; Mon, 24 Jul 2006 15:28:24 +0000 (GMT) (envelope-from rees@citi.umich.edu) Received: from citi.umich.edu (dumaguete.citi.umich.edu [141.211.133.51]) by citi.umich.edu (Postfix) with ESMTP id C32D81BB93; Mon, 24 Jul 2006 11:28:23 -0400 (EDT) From: Jim Rees To: rick@snowhite.cis.uoguelph.ca In-Reply-To: <200607241456.KAA63411@snowhite.cis.uoguelph.ca> (rick@snowhite.cis.uoguelph.ca, Mon, 24 Jul 2006 10:56:50 EDT) Date: Mon, 24 Jul 2006 11:28:23 -0400 Sender: rees@citi.umich.edu Message-Id: <20060724152823.C32D81BB93@citi.umich.edu> Cc: fs@freebsd.org Subject: Re: freebsd4.11 patch for nfs over tcp X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jul 2006 15:28:24 -0000 Delayed acks when properly implemented should not cause a performance hit. The problem is that there are many buggy implementations. If you have to turn off delayed acks to get good nfs performance, then there is a bug in the tcp stack. See, for example, rfc 2525 page 40, "Stretch ACK violation." MacOS has this same problem. Having said that, the code is obviously wrong. It's trying to set NODELAY and failing. This has apparently been fixed, because 6.1 does not have this bug. I'm pretty sure 4.x is out of maintenance. From owner-freebsd-fs@FreeBSD.ORG Mon Jul 24 18:52:06 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8290B16A4DA for ; Mon, 24 Jul 2006 18:52:06 +0000 (UTC) (envelope-from adamsch1@yahoo.com) Received: from web31812.mail.mud.yahoo.com (web31812.mail.mud.yahoo.com [68.142.207.75]) by mx1.FreeBSD.org (Postfix) with SMTP id 1370043D45 for ; Mon, 24 Jul 2006 18:52:06 +0000 (GMT) (envelope-from adamsch1@yahoo.com) Received: (qmail 17027 invoked by uid 60001); 24 Jul 2006 18:52:05 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=0R1jOx9ZQ2XYs6qozfRdYls6CKkUYfSZrHJYvV8Ov0pByxMQVJUe518IurEcRtqu8Ibc37pVB4sLd7SlxuCrVf5PJnmPu2aRaw6LzUhCC7KesIK2PTpiMuF6sPt1v3WIleN5lYCrobvuJmNoccVAdStT+RWCwoIfTSfwT8yPdEk= ; Message-ID: <20060724185205.17021.qmail@web31812.mail.mud.yahoo.com> Received: from [207.126.230.225] by web31812.mail.mud.yahoo.com via HTTP; Mon, 24 Jul 2006 11:52:05 PDT Date: Mon, 24 Jul 2006 11:52:05 -0700 (PDT) From: Shane Adams To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Shane Adams List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jul 2006 18:52:06 -0000 Hello, I'm a programmer that is new to kernel development. I'm interested in hacking around on UFS and FFS. I only have one machine so I copied the complete sys/ufs code to a new directory under fs, and changed a few things to get it to compile and mount. Everything works, and I was ready to start hacking without worrying so much about screwing up my system. Naturally I've rebooted the machine a few times since then, and I was wondering if anyone has advice for a fledgling kernel programmer. (Best practices) I read someplace that the UFS (some revision of the code) was written in userland then ported to the kernel. Is that how McKusick implemented softupdates or snapshots? The code is so dense I have trouble fathoming writing that much code in the mannor in which I've been tackling it. When I make a mistake, the machine freeze's, then reboots. I don't seem to get a kernel dump (or I'm not looking in the right location). Anway, kind of a broad question I know, just looking for a few tips! Cheers, Shane From owner-freebsd-fs@FreeBSD.ORG Mon Jul 24 19:06:38 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 361DF16A4DA for ; Mon, 24 Jul 2006 19:06:38 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8C39843D53 for ; Mon, 24 Jul 2006 19:06:37 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k6OJ6a6v054533; Mon, 24 Jul 2006 14:06:36 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <44C51A47.20801@centtech.com> Date: Mon, 24 Jul 2006 14:06:47 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.4 (X11/20060612) MIME-Version: 1.0 To: Shane Adams References: <20060724185205.17021.qmail@web31812.mail.mud.yahoo.com> In-Reply-To: <20060724185205.17021.qmail@web31812.mail.mud.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1615/Mon Jul 24 12:12:00 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jul 2006 19:06:38 -0000 On 07/24/06 13:52, Shane Adams wrote: > Hello, > > I'm a programmer that is new to kernel development. I'm interested in hacking around on UFS and FFS. I only have one machine so I copied the complete sys/ufs code to a new directory under fs, and changed a few things to get it to compile and mount. Everything works, and I was ready to start hacking without worrying so much about screwing up my system. > > Naturally I've rebooted the machine a few times since then, and I was wondering if anyone has advice for a fledgling kernel programmer. (Best practices) > > I read someplace that the UFS (some revision of the code) was written in userland then ported to the kernel. Is that how McKusick implemented softupdates or snapshots? The code is so dense I have trouble fathoming writing that much code in the mannor in which I've been tackling it. > > When I make a mistake, the machine freeze's, then reboots. I don't seem to get a kernel dump (or I'm not looking in the right location). > > Anway, kind of a broad question I know, just looking for a few tips! > > Cheers, > Shane Hi Shane! I'm in the same situation, and what I've done, is have a local copy of the cvs repo, then I check out (and/or update) a working copy into a separate directory, work on it, make changes, etc. Then I build (or build world, or whatever I need), and install to a mdconfig'ed disk image. Then, I boot qemu using that image, and run my tests, do debugging from the debugger, etc. I also nfs mount (using tap devices) the /usr/src in qemu from my separate checked out src tree on my local machine, so I can boot the qemu machine, and do buildworlds inside it (although it's slower). This seem to work very well for me, and I do all my development on my laptop while on-the-go. I too am tinkering with filesystems. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon Jul 24 19:11:44 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6D5D516A4DA for ; Mon, 24 Jul 2006 19:11:44 +0000 (UTC) (envelope-from adamsch1@yahoo.com) Received: from web31814.mail.mud.yahoo.com (web31814.mail.mud.yahoo.com [68.142.206.167]) by mx1.FreeBSD.org (Postfix) with SMTP id 7A77E43D67 for ; Mon, 24 Jul 2006 19:11:43 +0000 (GMT) (envelope-from adamsch1@yahoo.com) Received: (qmail 68909 invoked by uid 60001); 24 Jul 2006 19:11:39 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=ZZeQLpknGfpNulr4e+tBf3KjaeJsCjzTQV1/rZAVdSkL6GJQ4s2wl94L2XfMOBwzlEJ/vxP1SUX9r9zsyRS1f12T/hb1pExp/cDIclNmPUGd9G6K+xD6krpMyAJHeHEg93dJixqdKFWTJat0AyfsSPdp1fYv0rYnndrlGOa+aBM= ; Message-ID: <20060724191139.68907.qmail@web31814.mail.mud.yahoo.com> Received: from [207.126.230.225] by web31814.mail.mud.yahoo.com via HTTP; Mon, 24 Jul 2006 12:11:39 PDT Date: Mon, 24 Jul 2006 12:11:39 -0700 (PDT) From: Shane Adams To: Eric Anderson In-Reply-To: <44C51A47.20801@centtech.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-fs@freebsd.org Subject: Re: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Shane Adams List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jul 2006 19:11:44 -0000 Ah that makes sense, using an emulator so you can remote debug and step through the code. Nice. I had not heard of qemu, but I will check it out and see if i can get a similar setup working. nice suggestions thank you! I have debian running under vmware on my windows laptop, I wonder if I can remote debug what that platform. Something to look into. Cheers, Shane ----- Original Message ---- From: Eric Anderson To: Shane Adams Cc: freebsd-fs@freebsd.org Sent: Monday, July 24, 2006 12:06:47 PM Subject: Re: Advice for hacking on ufs/ffs On 07/24/06 13:52, Shane Adams wrote: > Hello, > > I'm a programmer that is new to kernel development. I'm interested in hacking around on UFS and FFS. I only have one machine so I copied the complete sys/ufs code to a new directory under fs, and changed a few things to get it to compile and mount. Everything works, and I was ready to start hacking without worrying so much about screwing up my system. > > Naturally I've rebooted the machine a few times since then, and I was wondering if anyone has advice for a fledgling kernel programmer. (Best practices) > > I read someplace that the UFS (some revision of the code) was written in userland then ported to the kernel. Is that how McKusick implemented softupdates or snapshots? The code is so dense I have trouble fathoming writing that much code in the mannor in which I've been tackling it. > > When I make a mistake, the machine freeze's, then reboots. I don't seem to get a kernel dump (or I'm not looking in the right location). > > Anway, kind of a broad question I know, just looking for a few tips! > > Cheers, > Shane Hi Shane! I'm in the same situation, and what I've done, is have a local copy of the cvs repo, then I check out (and/or update) a working copy into a separate directory, work on it, make changes, etc. Then I build (or build world, or whatever I need), and install to a mdconfig'ed disk image. Then, I boot qemu using that image, and run my tests, do debugging from the debugger, etc. I also nfs mount (using tap devices) the /usr/src in qemu from my separate checked out src tree on my local machine, so I can boot the qemu machine, and do buildworlds inside it (although it's slower). This seem to work very well for me, and I do all my development on my laptop while on-the-go. I too am tinkering with filesystems. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon Jul 24 19:42:13 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3E78316A4DE for ; Mon, 24 Jul 2006 19:42:13 +0000 (UTC) (envelope-from adamsch1@yahoo.com) Received: from web31804.mail.mud.yahoo.com (web31804.mail.mud.yahoo.com [68.142.207.67]) by mx1.FreeBSD.org (Postfix) with SMTP id 9990743D5C for ; Mon, 24 Jul 2006 19:42:09 +0000 (GMT) (envelope-from adamsch1@yahoo.com) Received: (qmail 55039 invoked by uid 60001); 24 Jul 2006 19:42:08 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=nfOPYiEuwWpSVhy32Nyj+vAA5ziNffTLJGp1DongB+HXdEHTNYpQl+lSu1jy41SzXQ1mwLRs5qwqf1bXC4lnCjnC6+w7uMahFb0tUpsrZIrW+ESZUG199dnrsU03aKub7VryBLtsubDtGrt9v7YbnqQx93Bu/n7zdOu9sy3hyec= ; Message-ID: <20060724194208.55037.qmail@web31804.mail.mud.yahoo.com> Received: from [207.126.230.225] by web31804.mail.mud.yahoo.com via HTTP; Mon, 24 Jul 2006 12:42:08 PDT Date: Mon, 24 Jul 2006 12:42:08 -0700 (PDT) From: Shane Adams To: Eric Anderson In-Reply-To: <44C51A47.20801@centtech.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-fs@freebsd.org Subject: Re: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Shane Adams List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jul 2006 19:42:13 -0000 Quick question - how do you install to a disk image? I'm still at work. I myself use mdconfig to create a file backed device that I use to mount and experiment with. I just newfs on that device and away I go. I was thinking of getting the small 20mb boot image and using that as a base. Just need some clarification on how to install my module on top of that. Cheers, Shane ----- Original Message ---- From: Eric Anderson To: Shane Adams Cc: freebsd-fs@freebsd.org Sent: Monday, July 24, 2006 12:06:47 PM Subject: Re: Advice for hacking on ufs/ffs On 07/24/06 13:52, Shane Adams wrote: > Hello, > > I'm a programmer that is new to kernel development. I'm interested in hacking around on UFS and FFS. I only have one machine so I copied the complete sys/ufs code to a new directory under fs, and changed a few things to get it to compile and mount. Everything works, and I was ready to start hacking without worrying so much about screwing up my system. > > Naturally I've rebooted the machine a few times since then, and I was wondering if anyone has advice for a fledgling kernel programmer. (Best practices) > > I read someplace that the UFS (some revision of the code) was written in userland then ported to the kernel. Is that how McKusick implemented softupdates or snapshots? The code is so dense I have trouble fathoming writing that much code in the mannor in which I've been tackling it. > > When I make a mistake, the machine freeze's, then reboots. I don't seem to get a kernel dump (or I'm not looking in the right location). > > Anway, kind of a broad question I know, just looking for a few tips! > > Cheers, > Shane Hi Shane! I'm in the same situation, and what I've done, is have a local copy of the cvs repo, then I check out (and/or update) a working copy into a separate directory, work on it, make changes, etc. Then I build (or build world, or whatever I need), and install to a mdconfig'ed disk image. Then, I boot qemu using that image, and run my tests, do debugging from the debugger, etc. I also nfs mount (using tap devices) the /usr/src in qemu from my separate checked out src tree on my local machine, so I can boot the qemu machine, and do buildworlds inside it (although it's slower). This seem to work very well for me, and I do all my development on my laptop while on-the-go. I too am tinkering with filesystems. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon Jul 24 19:50:45 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 33D2C16A4DA for ; Mon, 24 Jul 2006 19:50:45 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1EBA443D66 for ; Mon, 24 Jul 2006 19:50:43 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id k6OJohSH044611; Mon, 24 Jul 2006 14:50:43 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <44C5249E.4010603@centtech.com> Date: Mon, 24 Jul 2006 14:50:54 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.4 (X11/20060612) MIME-Version: 1.0 To: Shane Adams References: <20060724194208.55037.qmail@web31804.mail.mud.yahoo.com> In-Reply-To: <20060724194208.55037.qmail@web31804.mail.mud.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1616/Mon Jul 24 12:49:29 2006 on mh2.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jul 2006 19:50:45 -0000 On 07/24/06 14:42, Shane Adams wrote: > Quick question - how do you install to a disk image? > > I'm still at work. I myself use mdconfig to create a file backed device that I use to mount and experiment with. I just newfs on that device and away I go. > > I was thinking of getting the small 20mb boot image and using that as a base. Just need some clarification on how to install my module on top of that. > > Cheers, > Shane I use mdconfig and then do an installworld with DESTDIR and such set. man build should help with that I think. I also sent you my personal cheat sheet on this if that helps. Eric > ----- Original Message ---- > From: Eric Anderson > To: Shane Adams > Cc: freebsd-fs@freebsd.org > Sent: Monday, July 24, 2006 12:06:47 PM > Subject: Re: Advice for hacking on ufs/ffs > > On 07/24/06 13:52, Shane Adams wrote: >> Hello, >> >> I'm a programmer that is new to kernel development. I'm interested in hacking around on UFS and FFS. I only have one machine so I copied the complete sys/ufs code to a new directory under fs, and changed a few things to get it to compile and mount. Everything works, and I was ready to start hacking without worrying so much about screwing up my system. >> >> Naturally I've rebooted the machine a few times since then, and I was wondering if anyone has advice for a fledgling kernel programmer. (Best practices) >> >> I read someplace that the UFS (some revision of the code) was written in userland then ported to the kernel. Is that how McKusick implemented softupdates or snapshots? The code is so dense I have trouble fathoming writing that much code in the mannor in which I've been tackling it. >> >> When I make a mistake, the machine freeze's, then reboots. I don't seem to get a kernel dump (or I'm not looking in the right location). >> >> Anway, kind of a broad question I know, just looking for a few tips! >> >> Cheers, >> Shane > > > Hi Shane! I'm in the same situation, and what I've done, is have a > local copy of the cvs repo, then I check out (and/or update) a working > copy into a separate directory, work on it, make changes, etc. Then I > build (or build world, or whatever I need), and install to a mdconfig'ed > disk image. Then, I boot qemu using that image, and run my tests, do > debugging from the debugger, etc. I also nfs mount (using tap devices) > the /usr/src in qemu from my separate checked out src tree on my local > machine, so I can boot the qemu machine, and do buildworlds inside it > (although it's slower). > > This seem to work very well for me, and I do all my development on my > laptop while on-the-go. I too am tinkering with filesystems. > > Eric > > -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon Jul 24 23:13:16 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9829A16A4DE for ; Mon, 24 Jul 2006 23:13:16 +0000 (UTC) (envelope-from fbsd-fs@mawer.org) Received: from mail-ihug.icp-qv1-irony6.iinet.net.au (ihug-mail.icp-qv1-irony6.iinet.net.au [203.59.1.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id EB2DF43D58 for ; Mon, 24 Jul 2006 23:13:15 +0000 (GMT) (envelope-from fbsd-fs@mawer.org) Received: from 203-206-173-235.perm.iinet.net.au (HELO [127.0.0.1]) ([203.206.173.235]) by mail-ihug.icp-qv1-irony6.iinet.net.au with ESMTP; 25 Jul 2006 07:13:14 +0800 X-BrightmailFiltered: true X-Brightmail-Tracker: AAAAAA== X-IronPort-AV: i="4.07,177,1151856000"; d="scan'208"; a="390036612:sNHT73992884" Message-ID: <44C55407.2090308@mawer.org> Date: Tue, 25 Jul 2006 09:13:11 +1000 From: Antony Mawer User-Agent: Thunderbird 1.5.0.4 (Windows/20060516) MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: NWFS cleanup - anyone for a bounty? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jul 2006 23:13:16 -0000 Hi all, The NWFS code in FreeBSD 6.0 (and presumably 6.1) works fine in 95% of cases, but there are a few rough edges and problems that would be very nice if someone with more kernel/filesystem knowledge than myself could step up and investigate. A number of problems that have been fixed are similar to problems that have existed in the SMBFS code, and may simply need similar patches applied -- one example is this PR: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/59211 (I'm planning on testing this patch soon so it can be committed). Another unusual issue is where the "." directory in the mount point simply "vanishes", causing problems with directory enumeration: http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/65920 Are those who are also using NWFS interested in sponsoring someone to look at these problems, and is anyone interested in doing the work? I'm looking at what I can do to provide a reasonable test environment to anyone interested in working remotely on this (ie. access to a Netware server and a machine with a serial console to work on)... Cheers Antony From owner-freebsd-fs@FreeBSD.ORG Tue Jul 25 08:54:32 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 338FB16A4DD for ; Tue, 25 Jul 2006 08:54:32 +0000 (UTC) (envelope-from m.ehinger@ltur.de) Received: from postx.gateway-inter.net (postx.gateway-inter.net [213.144.19.80]) by mx1.FreeBSD.org (Postfix) with ESMTP id E8A1B43D46 for ; Tue, 25 Jul 2006 08:54:31 +0000 (GMT) (envelope-from m.ehinger@ltur.de) To: freebsd-fs@freebsd.org Message-ID: From: m.ehinger@ltur.de Date: Tue, 25 Jul 2006 10:54:20 +0200 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Subject: RE: NWFS cleanup - anyone for a bounty? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jul 2006 08:54:32 -0000 hi, i'am willing to help. At least i can test new patches. Regards, maik From owner-freebsd-fs@FreeBSD.ORG Tue Jul 25 10:09:45 2006 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5A36716A4DA for ; Tue, 25 Jul 2006 10:09:45 +0000 (UTC) (envelope-from peadar.edwards@gmail.com) Received: from wr-out-0506.google.com (wr-out-0506.google.com [64.233.184.230]) by mx1.FreeBSD.org (Postfix) with ESMTP id E776643D45 for ; Tue, 25 Jul 2006 10:09:44 +0000 (GMT) (envelope-from peadar.edwards@gmail.com) Received: by wr-out-0506.google.com with SMTP id i23so767120wra for ; Tue, 25 Jul 2006 03:09:44 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=OhiQgpXVlHm/BZl/dqBhkFUWgSXH3kJqMoeH4lYnnzh6IPPp3etUyy8CS5GFbbjmDOSKp0NGJ2VnBvy47DhIyB/q7iirCzoEq9iew08PJ0vMXA4ShG21HJKhRVl6wpztulSHFnG4c8SmD097ZOZL1Q8/14G5zb8D8zd0sIQ+heQ= Received: by 10.64.21.6 with SMTP id 6mr4943402qbu; Tue, 25 Jul 2006 03:09:43 -0700 (PDT) Received: by 10.65.139.8 with HTTP; Tue, 25 Jul 2006 03:09:43 -0700 (PDT) Message-ID: <34cb7c840607250309t1b20179icb034cad7e720e7f@mail.gmail.com> Date: Tue, 25 Jul 2006 11:09:43 +0100 From: "Peter Edwards" To: "Jim Rees" In-Reply-To: <20060724152823.C32D81BB93@citi.umich.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <200607241456.KAA63411@snowhite.cis.uoguelph.ca> <20060724152823.C32D81BB93@citi.umich.edu> Cc: fs@freebsd.org, rick@snowhite.cis.uoguelph.ca Subject: Re: freebsd4.11 patch for nfs over tcp X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jul 2006 10:09:45 -0000 On 7/24/06, Jim Rees wrote: > Delayed acks when properly implemented should not cause a performance hit. > The problem is that there are many buggy implementations. If you have to > turn off delayed acks to get good nfs performance, then there is a bug in > the tcp stack. See, for example, rfc 2525 page 40, "Stretch ACK violation." > MacOS has this same problem. > "NODELAY" is to turn of the Nagle algorithm, not delayed acks, and Nagle can interact badly with delayed acks on the peer for some workloads/protocols. I'm not sure NFS is one of those cases, but I suspect it probably is. From owner-freebsd-fs@FreeBSD.ORG Tue Jul 25 13:59:46 2006 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 48A0316A4DD for ; Tue, 25 Jul 2006 13:59:46 +0000 (UTC) (envelope-from rees@citi.umich.edu) Received: from citi.umich.edu (citi.umich.edu [141.211.133.111]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1876843D45 for ; Tue, 25 Jul 2006 13:59:46 +0000 (GMT) (envelope-from rees@citi.umich.edu) Received: from citi.umich.edu (dsl093-001-248.det1.dsl.speakeasy.net [66.93.1.248]) by citi.umich.edu (Postfix) with ESMTP id 94DC81BBF0; Tue, 25 Jul 2006 09:59:45 -0400 (EDT) From: Jim Rees To: "Peter Edwards" In-Reply-To: <34cb7c840607250309t1b20179icb034cad7e720e7f@mail.gmail.com> ("Peter Edwards", Tue, 25 Jul 2006 11:09:43 BST) Date: Tue, 25 Jul 2006 09:59:45 -0400 Sender: rees@citi.umich.edu Message-Id: <20060725135945.94DC81BBF0@citi.umich.edu> Cc: fs@freebsd.org, rick@snowhite.cis.uoguelph.ca Subject: Re: freebsd4.11 patch for nfs over tcp X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jul 2006 13:59:46 -0000 I see by inspecting the code that you are right and the web site I consulted was wrong. Don't believe everything you read on the web. I also see that Nagle himself suggests not using his algorithm with delayed acks. It can work, but all the timers have to be just right. Anyway this bug was fixed a while ago, and I don't think there will be another 4.x kernel, so there's not much to be done about this. From owner-freebsd-fs@FreeBSD.ORG Tue Jul 25 14:25:42 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E75D416A4E1 for ; Tue, 25 Jul 2006 14:25:41 +0000 (UTC) (envelope-from yar@comp.chem.msu.su) Received: from comp.chem.msu.su (comp.chem.msu.su [158.250.32.97]) by mx1.FreeBSD.org (Postfix) with ESMTP id BD8D243D76 for ; Tue, 25 Jul 2006 14:25:33 +0000 (GMT) (envelope-from yar@comp.chem.msu.su) Received: from comp.chem.msu.su (localhost [127.0.0.1]) by comp.chem.msu.su (8.13.4/8.13.3) with ESMTP id k6PEPTNn069753; Tue, 25 Jul 2006 18:25:29 +0400 (MSD) (envelope-from yar@comp.chem.msu.su) Received: (from yar@localhost) by comp.chem.msu.su (8.13.4/8.13.3/Submit) id k6PEPSeX069752; Tue, 25 Jul 2006 18:25:28 +0400 (MSD) (envelope-from yar) Date: Tue, 25 Jul 2006 18:25:28 +0400 From: Yar Tikhiy To: freebsd-fs@freebsd.org Message-ID: <20060725142528.GH67926@comp.chem.msu.su> References: <20060626112839.GA12741@static.protection.cx> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060626112839.GA12741@static.protection.cx> User-Agent: Mutt/1.5.9i Cc: Subject: Re: Fix parsing of UDF type 1 partition maps X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jul 2006 14:25:42 -0000 On Mon, Jun 26, 2006 at 08:28:39AM -0300, Pedro Martelletto wrote: > In udf_find_partmaps(), when we find a type 1 partition map, we have to > skip the actual type 1 length (6 bytes). With this diff, it is possible > to correctly spot the VAT partition map in a couple of discs I've. All, JFYI: I went ahead and committed this patch from Pedro, who has been taking care of our UDF driver for quite a while now. Thank you, Pedro! > Thanks for your attention, > > -p. > > Index: ecma167-udf.h > =================================================================== > RCS file: /home/ncvs/src/sys/fs/udf/ecma167-udf.h,v > retrieving revision 1.6 > diff -u -p -r1.6 ecma167-udf.h > --- ecma167-udf.h 3 Feb 2006 15:25:52 -0000 1.6 > +++ ecma167-udf.h 26 Jun 2006 10:45:12 -0000 > @@ -201,8 +201,6 @@ struct logvol_desc { > uint8_t maps[1]; > } __packed; > > -#define UDF_PMAP_SIZE 64 > - > /* Type 1 Partition Map [3/10.7.2] */ > struct part_map_1 { > uint8_t type; > @@ -211,6 +209,8 @@ struct part_map_1 { > uint16_t part_num; > } __packed; > > +#define UDF_PMAP_TYPE1_SIZE 6 > + > /* Type 2 Partition Map [3/10.7.3] */ > struct part_map_2 { > uint8_t type; > @@ -218,6 +218,8 @@ struct part_map_2 { > uint8_t part_id[62]; > } __packed; > > +#define UDF_PMAP_TYPE2_SIZE 64 > + > /* Virtual Partition Map [UDF 2.01/2.2.8] */ > struct part_map_virt { > uint8_t type; > @@ -245,7 +247,6 @@ struct part_map_spare { > } __packed; > > union udf_pmap { > - uint8_t data[UDF_PMAP_SIZE]; > struct part_map_1 pm1; > struct part_map_2 pm2; > struct part_map_virt pmv; > Index: udf_vfsops.c > =================================================================== > RCS file: /home/ncvs/src/sys/fs/udf/udf_vfsops.c,v > retrieving revision 1.42 > diff -u -p -r1.42 udf_vfsops.c > --- udf_vfsops.c 26 Jun 2006 03:21:19 -0000 1.42 > +++ udf_vfsops.c 26 Jun 2006 10:45:12 -0000 > @@ -723,30 +723,31 @@ udf_vptofh (struct vnode *vp, struct fid > static int > udf_find_partmaps(struct udf_mnt *udfmp, struct logvol_desc *lvd) > { > - union udf_pmap *pmap; > struct part_map_spare *pms; > struct regid *pmap_id; > struct buf *bp; > unsigned char regid_id[UDF_REGID_ID_SIZE + 1]; > int i, k, ptype, psize, error; > + uint8_t *pmap = (uint8_t *) &lvd->maps[0]; > > for (i = 0; i < le32toh(lvd->n_pm); i++) { > - pmap = (union udf_pmap *)&lvd->maps[i * UDF_PMAP_SIZE]; > - ptype = pmap->data[0]; > - psize = pmap->data[1]; > + ptype = pmap[0]; > + psize = pmap[1]; > if (((ptype != 1) && (ptype != 2)) || > - ((psize != UDF_PMAP_SIZE) && (psize != 6))) { > + ((psize != UDF_PMAP_TYPE1_SIZE) && > + (psize != UDF_PMAP_TYPE2_SIZE))) { > printf("Invalid partition map found\n"); > return (1); > } > > if (ptype == 1) { > /* Type 1 map. We don't care */ > + pmap += UDF_PMAP_TYPE1_SIZE; > continue; > } > > /* Type 2 map. Gotta find out the details */ > - pmap_id = (struct regid *)&pmap->data[4]; > + pmap_id = (struct regid *)&pmap[4]; > bzero(®id_id[0], UDF_REGID_ID_SIZE); > bcopy(&pmap_id->id[0], ®id_id[0], UDF_REGID_ID_SIZE); > > @@ -756,7 +757,8 @@ udf_find_partmaps(struct udf_mnt *udfmp, > return (1); > } > > - pms = &pmap->pms; > + pms = (struct part_map_spare *)pmap; > + pmap += UDF_PMAP_TYPE2_SIZE; > MALLOC(udfmp->s_table, struct udf_sparing_table *, > le32toh(pms->st_size), M_UDFMOUNT, M_NOWAIT | M_ZERO); > if (udfmp->s_table == NULL) > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" -- Yar From owner-freebsd-fs@FreeBSD.ORG Tue Jul 25 14:53:17 2006 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4FA6C16A506 for ; Tue, 25 Jul 2006 14:53:17 +0000 (UTC) (envelope-from peadar.edwards@gmail.com) Received: from wr-out-0506.google.com (wr-out-0506.google.com [64.233.184.228]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4CF1943E92 for ; Tue, 25 Jul 2006 14:50:53 +0000 (GMT) (envelope-from peadar.edwards@gmail.com) Received: by wr-out-0506.google.com with SMTP id i23so807481wra for ; Tue, 25 Jul 2006 07:50:47 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=tV6oLKrTdK5h/WAWpjOXm9EW7wShZo8qOUy/LrJkxgwEEuTEk+Pl45BzcsvqO/v0M9yNisWl3Tx4UKS/3V0itwWPWedn6KDVLdWCxqqSfyBWIkPVQEnHveqc8YMT/JTXZdl/DOcFXSYyp3bOJZWvazvYeqUFlWVm/I32vd2z5uk= Received: by 10.64.3.9 with SMTP id 9mr4400321qbc; Tue, 25 Jul 2006 07:50:47 -0700 (PDT) Received: by 10.65.139.8 with HTTP; Tue, 25 Jul 2006 07:50:47 -0700 (PDT) Message-ID: <34cb7c840607250750u1590d371w6655cc60a8736be6@mail.gmail.com> Date: Tue, 25 Jul 2006 15:50:47 +0100 From: "Peter Edwards" To: "Jim Rees" In-Reply-To: <20060725135945.94DC81BBF0@citi.umich.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <34cb7c840607250309t1b20179icb034cad7e720e7f@mail.gmail.com> <20060725135945.94DC81BBF0@citi.umich.edu> Cc: fs@freebsd.org, rick@snowhite.cis.uoguelph.ca Subject: Re: freebsd4.11 patch for nfs over tcp X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jul 2006 14:53:17 -0000 On 7/25/06, Jim Rees wrote: > I see by inspecting the code that you are right and the web site I consulted > was wrong. Don't believe everything you read on the web. > :-) I always point people at "Rethinking the TCP Nagle Algorithm" (Mogul & Minshall) when this comes up. It's a nice readable paper on the issues and some propsed solutions. http://www.acm.org/sigs/sigcomm/ccr/archive/2001/jan01/ccr-200101-mogul.pdf > I also see that Nagle himself suggests not using his algorithm with delayed > acks. It can work, but all the timers have to be just right. > > Anyway this bug was fixed a while ago, and I don't think there will be > another 4.x kernel, so there's not much to be done about this. > Absolutely. just setting things straight "for the record." From owner-freebsd-fs@FreeBSD.ORG Tue Jul 25 15:08:33 2006 Return-Path: X-Original-To: fs@freebsd.org Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 94CC916A4DE for ; Tue, 25 Jul 2006 15:08:33 +0000 (UTC) (envelope-from rick@snowhite.cis.uoguelph.ca) Received: from ccshst09.cs.uoguelph.ca (ccshst09.cs.uoguelph.ca [131.104.94.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2EDE243D46 for ; Tue, 25 Jul 2006 15:08:30 +0000 (GMT) (envelope-from rick@snowhite.cis.uoguelph.ca) Received: from snowhite.cis.uoguelph.ca (snowhite.cis.uoguelph.ca [131.104.48.1]) by ccshst09.cs.uoguelph.ca (8.13.1/8.13.1) with ESMTP id k6PF8TZK030163; Tue, 25 Jul 2006 11:08:29 -0400 Received: (from rick@localhost) by snowhite.cis.uoguelph.ca (8.9.3/8.9.3) id LAA73358; Tue, 25 Jul 2006 11:09:41 -0400 (EDT) Date: Tue, 25 Jul 2006 11:09:41 -0400 (EDT) From: rick@snowhite.cis.uoguelph.ca Message-Id: <200607251509.LAA73358@snowhite.cis.uoguelph.ca> To: fs@freebsd.org X-Scanned-By: MIMEDefang 2.52 on 131.104.94.206 Cc: peadar.edwards@gmail.com Subject: Re: freebsd4.11 patch for nfs over tcp X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jul 2006 15:08:33 -0000 > I always point people at "Rethinking the TCP Nagle Algorithm" (Mogul & > Minshall) when this comes up. It's a nice readable paper on the issues > and some propsed solutions. And, yes, it does cause performance problems for NFS. If an RPC is in the send queue, it delays sending the request. For the case I was testing that normally takes 5 seconds, it took about 3 minutes. You could watch the server's disk led "blink , blink...". As an historical note, it was Tom Talpey who suggested turning off the Nagle algorithm to me in 1989, when I was observing slow NFS over TCP on a MicroVAXII (now, that means REALLY SLOOWWWW). rick From owner-freebsd-fs@FreeBSD.ORG Tue Jul 25 15:13:11 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2957416A4DD for ; Tue, 25 Jul 2006 15:13:11 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id A253D43D45 for ; Tue, 25 Jul 2006 15:13:10 +0000 (GMT) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id DD2612086; Tue, 25 Jul 2006 17:13:05 +0200 (CEST) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: 0.0/3.0 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on tim.des.no Received: from xps.des.no (des.no [80.203.243.180]) by tim.des.no (Postfix) with ESMTP id CF1112082; Tue, 25 Jul 2006 17:13:05 +0200 (CEST) Received: by xps.des.no (Postfix, from userid 1001) id A629433C31; Tue, 25 Jul 2006 17:13:05 +0200 (CEST) From: des@des.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) To: Shane Adams References: <20060724185205.17021.qmail@web31812.mail.mud.yahoo.com> Date: Tue, 25 Jul 2006 17:13:05 +0200 In-Reply-To: <20060724185205.17021.qmail@web31812.mail.mud.yahoo.com> (Shane Adams's message of "Mon, 24 Jul 2006 11:52:05 -0700 (PDT)") Message-ID: <86wta1en0e.fsf@xps.des.no> User-Agent: Gnus/5.110003 (No Gnus v0.3) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jul 2006 15:13:11 -0000 Shane Adams writes: > I'm a programmer that is new to kernel development. I'm interested > in hacking around on UFS and FFS. I only have one machine so I > copied the complete sys/ufs code to a new directory under fs, and > changed a few things to get it to compile and mount. Everything > works, and I was ready to start hacking without worrying so much > about screwing up my system. > > Naturally I've rebooted the machine a few times since then, and I > was wondering if anyone has advice for a fledgling kernel > programmer. (Best practices) Get a test machine with a PXE-capable network adapter, and set up your workstation as bootp / dhcp and NFS server. Having a dedicated test box will save time as you won't have to reboot your workstation to test your code, and setting it up diskless will help even further. A serial console cable will help even more, as it will save you from moving from one keyboard to another and will give you a scrollback of the test machine's console from which you can copy / paste error messages, backtraces etc. DES --=20 Dag-Erling Sm=F8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Tue Jul 25 16:12:58 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5564116A4DD for ; Tue, 25 Jul 2006 16:12:58 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 567FB43D70 for ; Tue, 25 Jul 2006 16:12:53 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k6PGCnWO057429; Tue, 25 Jul 2006 11:12:49 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <44C6430D.1050506@centtech.com> Date: Tue, 25 Jul 2006 11:13:01 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.4 (X11/20060612) MIME-Version: 1.0 To: =?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?= References: <20060724185205.17021.qmail@web31812.mail.mud.yahoo.com> <86wta1en0e.fsf@xps.des.no> In-Reply-To: <86wta1en0e.fsf@xps.des.no> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Scanned: ClamAV 0.87.1/1618/Mon Jul 24 20:12:40 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jul 2006 16:12:58 -0000 On 07/25/06 10:13, Dag-Erling Smørgrav wrote: > Shane Adams writes: >> I'm a programmer that is new to kernel development. I'm interested >> in hacking around on UFS and FFS. I only have one machine so I >> copied the complete sys/ufs code to a new directory under fs, and >> changed a few things to get it to compile and mount. Everything >> works, and I was ready to start hacking without worrying so much >> about screwing up my system. >> >> Naturally I've rebooted the machine a few times since then, and I >> was wondering if anyone has advice for a fledgling kernel >> programmer. (Best practices) > > Get a test machine with a PXE-capable network adapter, and set up your > workstation as bootp / dhcp and NFS server. Having a dedicated test > box will save time as you won't have to reboot your workstation to > test your code, and setting it up diskless will help even further. A > serial console cable will help even more, as it will save you from > moving from one keyboard to another and will give you a scrollback of > the test machine's console from which you can copy / paste error > messages, backtraces etc. > > DES DES - anything special with the nfs root'ed box, to make this work well? The only reason I didn't set this up for my work is because it isn't portable.. :) Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Jul 25 17:54:24 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B01BC16A4DE for ; Tue, 25 Jul 2006 17:54:24 +0000 (UTC) (envelope-from adamsch1@yahoo.com) Received: from web31806.mail.mud.yahoo.com (web31806.mail.mud.yahoo.com [68.142.207.69]) by mx1.FreeBSD.org (Postfix) with SMTP id 2D16343D95 for ; Tue, 25 Jul 2006 17:53:47 +0000 (GMT) (envelope-from adamsch1@yahoo.com) Received: (qmail 78304 invoked by uid 60001); 25 Jul 2006 17:53:31 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=O5sF4FLG4Gx5OtZ3lirebPYVWbaGwA5w1g3m86mjudfcU9fjn1jbK7Ko7tIdw+C+oDQjz4gENkaFO1/k7H6MzBzmvLG2fY1Pdr0Zudft1Cd90nQSYdDpN4o88tXKZr8zWz5CYuj2MPGU/Xw8EfNcKHQ6KCXKbdrrZfDjVW4fN3c= ; Message-ID: <20060725175331.78302.qmail@web31806.mail.mud.yahoo.com> Received: from [207.126.230.225] by web31806.mail.mud.yahoo.com via HTTP; Tue, 25 Jul 2006 10:53:31 PDT Date: Tue, 25 Jul 2006 10:53:31 -0700 (PDT) From: Shane Adams To: Eric Anderson , =?utf-8?q?Dag-Erling=20Sm=C3=B8rgrav?= In-Reply-To: <44C6430D.1050506@centtech.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Shane Adams List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jul 2006 17:54:24 -0000 Thanks for the tip DES! I will probably break down and buy a cheap machine= or dust off my old P3 box. I have a PXE card at home too. I want to try = qemu as well cause if I get it to work I can tool around on my laptop. Eric, I hate to bother you but I didn't get the cheat sheet. Here's what I= attempted last night. I built qemu from ports I used dd to create 1G file. I did mdconfig -a -t vnode -f /bigdrive/bsdfilesystem -u 0 I downloaded the 20M boot only ISO. I ran qemu specifying -cdrom as the 20M iso, and -hda to /bigdrive/bsdfiles= ystem. Qemu started fine, and I proceeded to go through the basic install process,= at that point it asked me to toss in the correct CD with the packages to c= omplete the installation (Which I didnt' have the CD handy and it was late = so I will try again tonight). Anyway - you said you do a make DESTDIR to your mdconfig'd file. Did you m= ount the file you mdconfiged? If so, does a make installworld install a bo= ot loader as well? I'm not clear on how you used qemu to boot agains that = device. Perhaps you did a normal BSD install on it then later mounted it t= o get your stuff on there? Cheers, Shane ----- Original Message ---- From: Eric Anderson To: Dag-Erling Sm=C3=B8rgrav Cc: Shane Adams ; freebsd-fs@freebsd.org Sent: Tuesday, July 25, 2006 9:13:01 AM Subject: Re: Advice for hacking on ufs/ffs On 07/25/06 10:13, Dag-Erling Sm=C3=B8rgrav wrote: > Shane Adams writes: >> I'm a programmer that is new to kernel development. I'm interested >> in hacking around on UFS and FFS. I only have one machine so I >> copied the complete sys/ufs code to a new directory under fs, and >> changed a few things to get it to compile and mount. Everything >> works, and I was ready to start hacking without worrying so much >> about screwing up my system. >> >> Naturally I've rebooted the machine a few times since then, and I >> was wondering if anyone has advice for a fledgling kernel >> programmer. (Best practices) >=20 > Get a test machine with a PXE-capable network adapter, and set up your > workstation as bootp / dhcp and NFS server. Having a dedicated test > box will save time as you won't have to reboot your workstation to > test your code, and setting it up diskless will help even further. A > serial console cable will help even more, as it will save you from > moving from one keyboard to another and will give you a scrollback of > the test machine's console from which you can copy / paste error > messages, backtraces etc. >=20 > DES DES - anything special with the nfs root'ed box, to make this work well?=20 The only reason I didn't set this up for my work is because it isn't=20 portable.. :) Eric --=20 ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Jul 25 18:34:11 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F0A5416A4DA for ; Tue, 25 Jul 2006 18:34:11 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (megan.kiwi-computer.com [63.224.10.3]) by mx1.FreeBSD.org (Postfix) with SMTP id 26B2A43D66 for ; Tue, 25 Jul 2006 18:34:10 +0000 (GMT) (envelope-from rick@kiwi-computer.com) Received: (qmail 32158 invoked by uid 2001); 25 Jul 2006 18:34:10 -0000 Date: Tue, 25 Jul 2006 13:34:10 -0500 From: "Rick C. Petty" To: Shane Adams Message-ID: <20060725183410.GA32063@megan.kiwi-computer.com> References: <44C6430D.1050506@centtech.com> <20060725175331.78302.qmail@web31806.mail.mud.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060725175331.78302.qmail@web31806.mail.mud.yahoo.com> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@freebsd.org Subject: Re: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd@kiwi-computer.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jul 2006 18:34:12 -0000 On Tue, Jul 25, 2006 at 10:53:31AM -0700, Shane Adams wrote: > > I used dd to create 1G file. A faster way is: touch filename truncate -s 1g filename You also may wish to look at qemu-img, which is installed with qemu. > I did mdconfig -a -t vnode -f /bigdrive/bsdfilesystem -u 0 I'm pretty sure "-t vnode" is implied by "-f". > I downloaded the 20M boot only ISO. > > I ran qemu specifying -cdrom as the 20M iso, and -hda to /bigdrive/bsdfilesystem. > > Qemu started fine, and I proceeded to go through the basic install process, at that point it asked me to toss in the correct CD with the packages to complete the installation (Which I didnt' have the CD handy and it was late so I will try again tonight). You didn't use the network install, especially if you only had the boot-only CD? If your media type was set to network, it should have attempted to grab the packages from there. Not that you need any packages installed to play with UFS. > Anyway - you said you do a make DESTDIR to your mdconfig'd file. Did you mount the file you mdconfiged? You have to mount it because DESTDIR points to a directory, not a device or image. I'd suggest: mount /dev/md0s1a /mnt [mount your other filesystems, if applicable, under /mnt/] cd /usr/src; make DESTDIR=/mnt installworld installkernel cd /usr/src/etc; make DESTDIR=/mnt distribution > If so, does a make installworld install a boot loader as well? No. > I'm not clear on how you used qemu to boot agains that device. Perhaps you did a normal BSD install on it then later mounted it to get your stuff on there? That would be easier, but you might wish to put an MBR and a label on your image: fdisk -BI /dev/md0 bsdlabel -Bw /dev/md0 You may wish to partition the label for separate file systems, although that's not necessary. Don't forget to newfs each of your filesystems, e.g. newfs /dev/md0s1a Using this method won't require you to run the installer inside qemu. The nice thing about using qemu to test your hacking is that you can use the -snapshot mode to preserve the disk image(s), that way if the qemu "box" panics, you don't need to fsck or any of that. -- Rick C. Petty From owner-freebsd-fs@FreeBSD.ORG Tue Jul 25 18:39:34 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BBD0B16A4DA for ; Tue, 25 Jul 2006 18:39:34 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2227043D49 for ; Tue, 25 Jul 2006 18:39:32 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id k6PIdEJp066452; Tue, 25 Jul 2006 13:39:14 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <44C6655E.6080807@centtech.com> Date: Tue, 25 Jul 2006 13:39:26 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.4 (X11/20060612) MIME-Version: 1.0 To: Shane Adams References: <20060725175331.78302.qmail@web31806.mail.mud.yahoo.com> In-Reply-To: <20060725175331.78302.qmail@web31806.mail.mud.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1618/Mon Jul 24 20:12:40 2006 on mh2.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org, =?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?= Subject: Re: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jul 2006 18:39:34 -0000 On 07/25/06 12:53, Shane Adams wrote: > Thanks for the tip DES! I will probably break down and buy a cheap machine or dust off my old P3 box. I have a PXE card at home too. I want to try qemu as well cause if I get it to work I can tool around on my laptop. > > Eric, I hate to bother you but I didn't get the cheat sheet. Here's what I attempted last night. > > I built qemu from ports > I used dd to create 1G file. > I did mdconfig -a -t vnode -f /bigdrive/bsdfilesystem -u 0 > I downloaded the 20M boot only ISO. > > I ran qemu specifying -cdrom as the 20M iso, and -hda to /bigdrive/bsdfilesystem. > > Qemu started fine, and I proceeded to go through the basic install process, at that point it asked me to toss in the correct CD with the packages to complete the installation (Which I didnt' have the CD handy and it was late so I will try again tonight). > > Anyway - you said you do a make DESTDIR to your mdconfig'd file. Did you mount the file you mdconfiged? If so, does a make installworld install a boot loader as well? I'm not clear on how you used qemu to boot agains that device. Perhaps you did a normal BSD install on it then later mounted it to get your stuff on there? I just resent it to you. I'll probably clean it up, and post it somewhere for anyone who cares at some point. I did it without sysinstall, because I wanted an updated version of the OS from my local checked out copy, which already had changes in it. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Jul 25 18:49:40 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5AFE716A4E7 for ; Tue, 25 Jul 2006 18:49:40 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1CD0B43D49 for ; Tue, 25 Jul 2006 18:49:34 +0000 (GMT) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id 7D56F2086; Tue, 25 Jul 2006 20:49:28 +0200 (CEST) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: 0.0/3.0 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on tim.des.no Received: from xps.des.no (des.no [80.203.243.180]) by tim.des.no (Postfix) with ESMTP id E0C7F2082; Tue, 25 Jul 2006 20:49:27 +0200 (CEST) Received: by xps.des.no (Postfix, from userid 1001) id BABE733C31; Tue, 25 Jul 2006 20:49:27 +0200 (CEST) From: des@des.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) To: Eric Anderson References: <20060724185205.17021.qmail@web31812.mail.mud.yahoo.com> <86wta1en0e.fsf@xps.des.no> <44C6430D.1050506@centtech.com> Date: Tue, 25 Jul 2006 20:49:27 +0200 In-Reply-To: <44C6430D.1050506@centtech.com> (Eric Anderson's message of "Tue, 25 Jul 2006 11:13:01 -0500") Message-ID: <86lkqheczs.fsf@xps.des.no> User-Agent: Gnus/5.110003 (No Gnus v0.3) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jul 2006 18:49:40 -0000 Eric Anderson writes: > DES - anything special with the nfs root'ed box, to make this work > well? Not really. Here's a short summary. Assume you have a spare disk in your workstation, ad1, which you are going to use for your test box. Create and populate a file system for your test box: % mkdir /box % echo '/dev/ad1 /box ufs rw 2 2' >>/etc/fstab % newfs -U /box % mount /box % mergemaster -i -D /box % cd /usr/src % make installworld DESTDIR=3D/box Check out a source tree: % cd /box/usr % cvs -R -q -d /home/ncvs co -P src Chroot into your sandbox: % echo 'devfs /box/dev devfs rw 0 0' >>/etc/fstab % mount /box/dev % chroot /box /bin/sh At this point you want to create an appropriate make.conf and kernel config for your test box. Useful make.conf settings include: # build *everything* with debugging symbols DEBUG_FLAGS ?=3D -g # and leave them in when installing STRIP =3D # empty # don't clean out your previous build before rebuilding the kernel - # you'll be doing that a lot NO_KERNELCLEAN =3D YES When you are ready, build and install world and kernel: [still inside the chroot] % cd /usr/src % make buildworld buildkernel && make installworld installkernel You don't really need to chroot; you can do everything outside the sandbox, making sure that DESTDIR and __MAKE_CONF are correctly defined. I find it easier to work in a chroot because it saves me from having to remember just that. If you're going to use a serial console, kick up the speed a notch: [still inside the chroot] % cat >>/boot/loader.conf <>/etc/fstab </usr/local/etc/dhcp.conf <>/etc/rc.conf % /usr/local/etc/rc.d/isc-dhcpd.sh start Of course, you also need to set up NFS and TFTP: [nfs] % echo 'nfs_server_enable=3D"YES"' >>/etc/rc.conf % echo 'mountd_enable=3D"YES"' >>/etc/rc.conf % echo '/box -maproot=3Droot -network 10.0.0.0 -mask 255.255.255.0' >>/etc/= exports % /etc/rc.d/nfsserver start % /etc/rc.d/mountd start [tftp] % mkdir /tftpboot % cp /box/boot/pxeboot /tftpboot % sed -i.bak -e 's/^#tftp/tftp/' /etc/inetd.conf % echo 'inetd_enable=3D"YES"' >>/etc/rc.conf % /etc/rc.d/inetd start At this point, if I didn't miss anything, you should be able to just switch on your test box and watch it boot. DES --=20 Dag-Erling Sm=F8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Thu Jul 27 07:14:46 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E6EA616A4DF for ; Thu, 27 Jul 2006 07:14:46 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7907543D4C for ; Thu, 27 Jul 2006 07:14:46 +0000 (GMT) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id 78F8E2085; Thu, 27 Jul 2006 09:14:42 +0200 (CEST) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: 0.0/3.0 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on tim.des.no Received: from xps.des.no (des.no [80.203.243.180]) by tim.des.no (Postfix) with ESMTP id 0115C2082; Thu, 27 Jul 2006 09:14:42 +0200 (CEST) Received: by xps.des.no (Postfix, from userid 1001) id D99D833C31; Thu, 27 Jul 2006 09:14:41 +0200 (CEST) From: des@des.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) To: rick-freebsd@kiwi-computer.com References: <44C6430D.1050506@centtech.com> <20060725175331.78302.qmail@web31806.mail.mud.yahoo.com> <20060725183410.GA32063@megan.kiwi-computer.com> Date: Thu, 27 Jul 2006 09:14:41 +0200 In-Reply-To: <20060725183410.GA32063@megan.kiwi-computer.com> (Rick C. Petty's message of "Tue, 25 Jul 2006 13:34:10 -0500") Message-ID: <86lkqf1pum.fsf@xps.des.no> User-Agent: Gnus/5.110003 (No Gnus v0.3) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jul 2006 07:14:47 -0000 "Rick C. Petty" writes: > On Tue, Jul 25, 2006 at 10:53:31AM -0700, Shane Adams wrote: > > I used dd to create 1G file. > A faster way is: > touch filename > truncate -s 1g filename no, this will result in a fragmented image, which will hurt run-time performance. DES --=20 Dag-Erling Sm=F8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Thu Jul 27 17:54:38 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 959EE16A4DE for ; Thu, 27 Jul 2006 17:54:38 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (megan.kiwi-computer.com [63.224.10.3]) by mx1.FreeBSD.org (Postfix) with SMTP id DAAF543D46 for ; Thu, 27 Jul 2006 17:54:37 +0000 (GMT) (envelope-from rick@kiwi-computer.com) Received: (qmail 48337 invoked by uid 2001); 27 Jul 2006 17:54:36 -0000 Date: Thu, 27 Jul 2006 12:54:36 -0500 From: "Rick C. Petty" To: Dag-Erling Sm?rgrav Message-ID: <20060727175436.GA48057@megan.kiwi-computer.com> References: <44C6430D.1050506@centtech.com> <20060725175331.78302.qmail@web31806.mail.mud.yahoo.com> <20060725183410.GA32063@megan.kiwi-computer.com> <86lkqf1pum.fsf@xps.des.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86lkqf1pum.fsf@xps.des.no> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@freebsd.org Subject: Re: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd@kiwi-computer.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jul 2006 17:54:38 -0000 On Thu, Jul 27, 2006 at 09:14:41AM +0200, Dag-Erling Sm?rgrav wrote: > "Rick C. Petty" writes: > > On Tue, Jul 25, 2006 at 10:53:31AM -0700, Shane Adams wrote: > > > I used dd to create 1G file. > > A faster way is: > > touch filename > > truncate -s 1g filename > > no, this will result in a fragmented image, which will hurt run-time > performance. Not necessarily. If you install a variant of windoze, the installer will format the disk first, writing zeros (and metadata) sequentially, and thus the allocation will not be fragmented. In the case of UFS newfs only the metadata will be written for each cylinder group and thus the logical blocks for all metadata will be contiguous. I wouldn't say this type of fragmenting absolutely *will* hurt performance but that it "may" hurt-- certainly tools like fsck would see a performance improvement. There are some performance advantages to reducing the seek time (in the underlying storage mechanism) for metadata operations, especially with larger disk caches. Without benchmarks, it's hard to say in general whether performance would be degraded due to the "random" allocation, if you also include FreeBSD's caching mechanisms. And UFS block allocation is quite predictable. But you are correct-- truncate(1) just changes the size of the file and doesn't allocate the underlying blocks. So there is a tendancy for it to become fragmented as blocks get allocated out of order. Also, I discovered that in my example, touch(1) isn't necessary... truncate(1) works on non-existent files. -- Rick C. Petty From owner-freebsd-fs@FreeBSD.ORG Fri Jul 28 09:27:55 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.ORG Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 050C016A4E0 for ; Fri, 28 Jul 2006 09:27:55 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3019D43D46 for ; Fri, 28 Jul 2006 09:27:53 +0000 (GMT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (fezkly@localhost [127.0.0.1]) by lurza.secnetix.de (8.13.4/8.13.4) with ESMTP id k6S9RR2J072246; Fri, 28 Jul 2006 11:27:32 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.13.4/8.13.1/Submit) id k6S9RG2B072245; Fri, 28 Jul 2006 11:27:16 +0200 (CEST) (envelope-from olli) Date: Fri, 28 Jul 2006 11:27:16 +0200 (CEST) Message-Id: <200607280927.k6S9RG2B072245@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG, rick-freebsd@kiwi-computer.com In-Reply-To: <20060727175436.GA48057@megan.kiwi-computer.com> X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.0-20051224 ("Ronay") (UNIX) (FreeBSD/4.11-STABLE (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Fri, 28 Jul 2006 11:27:47 +0200 (CEST) Cc: Subject: Re: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-fs@FreeBSD.ORG, rick-freebsd@kiwi-computer.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jul 2006 09:27:55 -0000 Rick C. Petty wrote: > Dag-Erling Smørgrav wrote: > > Rick C. Petty wrote: > > > Shane Adams wrote: > > > > I used dd to create 1G file. > > > A faster way is: > > > touch filename > > > truncate -s 1g filename > > > > no, this will result in a fragmented image, which will hurt run-time > > performance. Indeed. > Not necessarily. If you install a variant of windoze, the installer will > format the disk first, writing zeros (and metadata) sequentially, and > thus the allocation will not be fragmented. Not true. Last time I installed Windows, it did not overwrite the whole disk. If it did that, it would have taken _way_ too long (hours instead of minutes). > In the case of UFS newfs > only the metadata will be written for each cylinder group No. Only the superblock copies and block bitmaps are written during newfs, nothing else. > and thus the > logical blocks for all metadata will be contiguous. Nope. As soon as you start writing to such a file system, it will start allocating space for inodes, indirect block lists, directories and file data, which will probably lead to massive fragmentation. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. With Perl you can manipulate text, interact with programs, talk over networks, drive Web pages, perform arbitrary precision arithmetic, and write programs that look like Snoopy swearing. From owner-freebsd-fs@FreeBSD.ORG Fri Jul 28 14:36:17 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EFBF116A4DF for ; Fri, 28 Jul 2006 14:36:17 +0000 (UTC) (envelope-from user@dhp.com) Received: from shell.dhp.com (shell.dhp.com [199.245.105.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id ED7CB43D8B for ; Fri, 28 Jul 2006 14:35:40 +0000 (GMT) (envelope-from user@dhp.com) Received: by shell.dhp.com (Postfix, from userid 896) id 1E2DA3134B; Fri, 28 Jul 2006 10:35:10 -0400 (EDT) Date: Fri, 28 Jul 2006 10:35:10 -0400 (EDT) From: Ensel Sharon To: freebsd-fs@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: broken aaccli in FreeBSD 6.1 - linux version (afacli ?) needed... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jul 2006 14:36:18 -0000 Hello, The ports-standard aaccli works fine in FreeBSD 6.0-RELEASE (I am using a 2610sa). However the same ports-standard aaccli does not work in 6.1-RELEASE (I am using 2820SA cards). Apparently this aaccli is getting old, and we just crossed the threshold of functionality for it. Word on the street is that if I load the linux kernel modules and run the linux version of the tool (afacli, right ?) that it will work, as it is more up to date. The problem is, I cannot find this linux tool. Everything I download from adaptec is just a total disaster - tarballs full fof floppy images full fo god knows what. Can someone describe exactly which package I should be downloading from adaptec, and where in that package the linux binary is ? Previous threads on this topic have ended with "never mind, someone nice emailed it to me offlist" which is great, but for the sake of the archives, can someone respond and say which of the following: AACRAID RPM files b2420 for multiple Linux versions Linux Installation Driver Disk and Driver CD Images b2420 AACRAID driver patch files b2420 for multiple Linux Kernel versions Adaptec Storage Manager Version 4.10.4813 for Linux from: http://www.adaptec.com/en-US/downloads/rh/rhel_4?productId=AAR-2820SA&dn=Adaptec+Serial+ATA+II+RAID+2820SA which was linked to from: http://www.adaptec.com/en-US/support/sata/sataii/AAR-2820SA/ Thanks. From owner-freebsd-fs@FreeBSD.ORG Fri Jul 28 15:10:31 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.ORG Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 613A416A4DE for ; Fri, 28 Jul 2006 15:10:31 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (megan.kiwi-computer.com [63.224.10.3]) by mx1.FreeBSD.org (Postfix) with SMTP id BD55743D69 for ; Fri, 28 Jul 2006 15:10:30 +0000 (GMT) (envelope-from rick@kiwi-computer.com) Received: (qmail 58582 invoked by uid 2001); 28 Jul 2006 15:10:29 -0000 Date: Fri, 28 Jul 2006 10:10:29 -0500 From: "Rick C. Petty" To: freebsd-fs@FreeBSD.ORG Message-ID: <20060728151029.GB58252@megan.kiwi-computer.com> References: <20060727175436.GA48057@megan.kiwi-computer.com> <200607280927.k6S9RG2B072245@lurza.secnetix.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200607280927.k6S9RG2B072245@lurza.secnetix.de> User-Agent: Mutt/1.4.2.1i Cc: Subject: Re: Advice for hacking on ufs/ffs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd@kiwi-computer.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jul 2006 15:10:31 -0000 On Fri, Jul 28, 2006 at 11:27:16AM +0200, Oliver Fromme wrote: > Rick C. Petty wrote: > > > Not necessarily. If you install a variant of windoze, the installer will > > format the disk first, writing zeros (and metadata) sequentially, and > > thus the allocation will not be fragmented. > > Not true. Last time I installed Windows, it did not > overwrite the whole disk. If it did that, it would > have taken _way_ too long (hours instead of minutes). XP? I've certainly tried this with 2000 and it did precisely that. It certainly took hours to format 8+G NTFS partitions at install time. I'll check this out again today in qemu. Certainly qemu formats are significantly faster than native formats, due to FreeBSD's caching mechanisms. > > In the case of UFS newfs > > only the metadata will be written for each cylinder group > > No. Only the superblock copies and block bitmaps are > written during newfs, nothing else. For each cylinder group, the following is written: the superblock, the cylinder group map (summaries + block free bitmap), and "two blocks worth of inodes" (for UFS2.. in UFS1, all inode blocks are written). What else do you consider is the "metadata" for a filesystem? > > and thus the > > logical blocks for all metadata will be contiguous. > > Nope. As soon as you start writing to such a file system, > it will start allocating space for inodes, indirect block > lists, directories and file data, which will probably lead > to massive fragmentation. Things like indirect blocks will be allocated near the rest of the blocks for that file. So in allocation order: the first 12 direct blocks, the first indirect block, the blocks for that indirect block, rinse, repeat. Directories are just files, but they get pretty fragmented anyway (if the size of the directory, in bytes, is larger than one file system block, which by default is 16k). I wasn't disputing the fragmentation of these files but the performance of having the blocks of the underlying file-store allocated in the same order that UFS allocates them. It would be interesting to test the performance difference. -- Rick C. Petty