From owner-freebsd-current@FreeBSD.ORG Mon Jan 7 22:11:49 2008 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E57F816A417 for ; Mon, 7 Jan 2008 22:11:49 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id B82F013C442 for ; Mon, 7 Jan 2008 22:11:49 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.1/8.13.7) with ESMTP id m07MBfN5032226; Mon, 7 Jan 2008 14:11:41 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.14.1/8.13.4/Submit) id m07MBcb3032223; Mon, 7 Jan 2008 14:11:38 -0800 (PST) Date: Mon, 7 Jan 2008 14:11:38 -0800 (PST) From: Matthew Dillon Message-Id: <200801072211.m07MBcb3032223@apollo.backplane.com> To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= References: <200801012116.m01LGQhN012860@bonkers.video-collage.com> <200801032334.m03NY7Zd019292@apollo.backplane.com> <863at97ntr.fsf@ds4.des.no> Cc: Mikhail Teterin , efinleywork@efinley.com, current@freebsd.org Subject: Re: a new way to hang 7.0 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2008 22:11:50 -0000 :OK, so you reject softupdates because it took time to mature and you :assume it stopped improving when you stopped paying attention. : :How long do you think it will take for HAMMER to mature? Realistically? :How long will HAMMER be "a huge source of bugs in the system" before it :stabilizes? : :Think back to when you started DragonFly. How soon did you expect it to :overtake FreeBSD in SMP performance? And how long did it actually take? :Actually, it never happened - DrangonFly doesn't scale at all across :multiple cores, while FreeBSD 7 leads the pack. : :Perhaps you should adjust your expectations a bit. I don't doubt that :HAMMER will be a very interesting file system when it's stable, but I :doubt very much that will happen any time soon. In fact, I think it :will take about as long for HAMMER to mature as it took for softupdates :and SMPng. : :DES :--=20 :Dag-Erling Sm=C3=B8rgrav - des@des.no Don't mistake the existance of the MP lock for a lack of SMP coding. All kernel coding done in DragonFly these days is SMP oriented because all the APIs are SMP oriented, whether the MP lock is held or not. We are better positioned there then you think we are. If I am overly conservative when it comes to maintaining system stability, well, that's just a quirk of mine. I don't feel there's much of a point to having cool bells and whistles if it also means getting crashes, or introducing untraceable and difficult-to-debug bugs. Personally speaking, if I had the chance to inherit FreeBSD's MP work, I wouldn't touch it with a ten foot poll. Regardless of the performance you are getting out of it, your code base is a huge mess and it looks completely unmanagable to me. You have had to deal with a continuous stream of bugs from the same subsystems for the last, what, five years? I attribute that directly to the ridiculous amount of complexity you have introduced to all levels of the kernel. No thanks. With regards to softupdates verses HAMMER, I think you are trying to compare apples to oranges here. Softupdates still has bugs because it is very, VERY complex and fragile code that only three people in the entire world understands well enough to work on. HAMMER development is bounded only by its from-scratch implementation. It is extremely well organized, extremely robust, well commented, and the bugs are a short-lived byproduct of development. I expect it will become production ready very, very quickly once the remaining core work is completed (the on-the-fly recovery code and the long-term balancing code)... just like every other major application I've written over the years has been. Kirk stopped working on softupdates years ago, the snapshot code is severely limited, and it hasn't removed the need for fsck. Background fsck is still as dangerous to run as the day it was introduced, and the only softupdates work I see are attempts to fix bugs. From my point of view softupdates is dead, and UFS2 is in no better shape if you can't fix the fsck issue (and I have grave doubts about its ability to scaleh given the linear nature of most of the cluster algorithms). On top of that UFS has the same problems that it has always had. The dirhash code is a bandaid at best. I occassionally see people talking about building a log into UFS or resurrecting LFS. It's possible to do but it would also be the end of the line for the filesystem. The last gasp, so to speak. I have followed every single commit made to softupdates since its inception. Don't kid yourself. The major goal for DragonFly has been and always will be transparent clustering. HAMMER and its transaction oriented record storage abstraction is one of three major components needed to being able to achieve that. Above and beyond that, HAMMER solves at least half a dozen major infrastructure needs and it does it very cleanly. It had better, it took me all of 2007 to design the sucker :-). -Matt Matthew Dillon