From owner-freebsd-stable@FreeBSD.ORG Fri Feb 9 23:18:33 2007 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A8D4216A403 for ; Fri, 9 Feb 2007 23:18:33 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (gate.funkthat.com [69.17.45.168]) by mx1.freebsd.org (Postfix) with ESMTP id 64BAE13C47E for ; Fri, 9 Feb 2007 23:18:33 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (ijj8dl6skcs4o6ko@localhost.funkthat.com [127.0.0.1]) by hydrogen.funkthat.com (8.13.6/8.13.3) with ESMTP id l19NIVBR009707; Fri, 9 Feb 2007 15:18:31 -0800 (PST) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.13.6/8.13.3/Submit) id l19NITO4009706; Fri, 9 Feb 2007 15:18:29 -0800 (PST) (envelope-from jmg) Date: Fri, 9 Feb 2007 15:18:29 -0800 From: John-Mark Gurney To: Sean Bryant Message-ID: <20070209231829.GJ1620@funkthat.com> Mail-Followup-To: Sean Bryant , Antony Mawer , freebsd-stable@freebsd.org, Dominic Marks References: <45C52C3E.8040204@elgia.com> <20070205101806.b45f4118.dom@helenmarks.co.uk> <45C7EC5F.2030108@cyberwang.net> <45C81A5B.1010608@mawer.org> <20070207055131.GC1620@funkthat.com> <45CCC676.9070507@cyberwang.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <45CCC676.9070507@cyberwang.net> User-Agent: Mutt/1.4.2.1i X-Operating-System: FreeBSD 5.4-RELEASE-p6 i386 X-PGP-Fingerprint: B7 EC EF F8 AE ED A7 31 96 7A 22 B3 D8 56 36 F4 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html Cc: freebsd-stable@freebsd.org, Dominic Marks Subject: Re: dd as an imaging solution. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: John-Mark Gurney List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Feb 2007 23:18:33 -0000 Sean Bryant wrote this message on Fri, Feb 09, 2007 at 14:07 -0500: > John-Mark Gurney wrote: > >Antony Mawer wrote this message on Tue, Feb 06, 2007 at 17:04 +1100: > >>On 6/02/2007 1:47 PM, Sean Bryant wrote: > >>>Dominic Marks wrote: > >>>>Check out G4U (NetBSD based) > >>>The only problem I can see here is that multiple parallel reads will > >>>have serious performance impacts, thus greatly increasing the cloning of > >>>the disk. > >>> > >>>The solution with dd, tee and netcat would just daisy chain the copy > >>>across the network which would be way faster. > >>Now all you need is G4U to operate in a multicast manner like Symantec > >>Ghost Corporate Edition, and your transfer speed wouldn't reduce with > >>each additional client (eg. 100mbps for 1 client, 50mbps each for 2 > >>clients, 33.3mbps each for 3 clients, ...) > > > >Add FEC to the multicast, and you can constantly stream the data, and > >not have to worry about dropped segments as much... > > > Can you explain this? FEC stands for Forward Error Correction... Check out: http://info.iet.unipi.it/~luigi/fec.html for some work that Luigi has done wrt FEC. I've even embedded his FEC library in the kernel w/o too much difficulty... Wikipedia also has an article on it... So the idea is that you multicast out the data broken up into x packets.. In addition to x packets, you also transmit y "parity" packets... As long as the end system receives any combination of x and y packets where the total unique packet count is x, you are able to reconstruct the data... You choose y based upon your expected packet drop rate.. This has the advantage that when you are transmitting the data, and a machine fails to receive a packet, you don't have to a) retrainsmit the packet, or b) wait till the same data packet comes along... Because you can replace the missed data packet w/ one of the parity packets to reconstruct the data... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."