From owner-freebsd-questions@FreeBSD.ORG Fri Oct 5 02:30:18 2012 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9FDED106566B for ; Fri, 5 Oct 2012 02:30:18 +0000 (UTC) (envelope-from mike.jeays@rogers.com) Received: from nm19-vm0.bullet.mail.sp2.yahoo.com (nm19-vm0.bullet.mail.sp2.yahoo.com [98.139.91.216]) by mx1.freebsd.org (Postfix) with SMTP id 6B5BC8FC08 for ; Fri, 5 Oct 2012 02:30:18 +0000 (UTC) Received: from [98.139.91.62] by nm19.bullet.mail.sp2.yahoo.com with NNFMP; 05 Oct 2012 02:28:05 -0000 Received: from [98.139.44.84] by tm2.bullet.mail.sp2.yahoo.com with NNFMP; 05 Oct 2012 02:28:05 -0000 Received: from [127.0.0.1] by omp1021.access.mail.sp2.yahoo.com with NNFMP; 05 Oct 2012 02:28:05 -0000 X-Yahoo-Newman-Id: 309991.71889.bm@omp1021.access.mail.sp2.yahoo.com Received: (qmail 23919 invoked from network); 5 Oct 2012 02:28:05 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=rogers.com; h=DKIM-Signature:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:Received:Date:From:To:Cc:Subject:Message-ID:In-Reply-To:References:X-Mailer:Mime-Version:Content-Type:Content-Transfer-Encoding; b=rV+dMDlVnqGtQXLSjlSTN8hPAkOohSP8oH+u/Roj2UaR4T4YI7SBP0XlKONpjZT35oaL+uqQCfOILTtj4zW1pISPClXoymiLCQmbS3cEo2iJ5bog9kByLInUd//48YggmbixKBFfBJuL+VlQrIYGQaKS1I1S9yapC4jVYbRtyMU= ; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rogers.com; s=s1024; t=1349404085; bh=hzZP4k9tLdd8gaOhgNfln19GlyU+6VhjJzixNjejI4s=; h=X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:Received:Date:From:To:Cc:Subject:Message-ID:In-Reply-To:References:X-Mailer:Mime-Version:Content-Type:Content-Transfer-Encoding; b=snddXbt4ZCK94E6a0lWngdUr1xeCjW8KCb4vbZpZR0DH3dYDu3aPbg9/PVKrOntGa1BECq/WmWUC/HEBDJAQfFl4DFwrujQ9Kr0QyqB7lWI0FsYpEzXIRnp/lI8tAn50CG8cUVthLHsVKYUU38wVI4wrColvCBQCOYxEiEigny4= X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: MSPiYM4VM1nNCDBemBmG00GlMFm_YO00lt__sd6VoVDlBHX E3A6dYPVibk6T7N_5KyW0giTFqvwqrvDMFNlOsxWkFpFWJanqkN9cyClXPWI v_RpitI2riGaIcUy0UyVet_5f4SPyur_0.bfH.tFVBYi.8Tru_PJqf8KBDqy wuZoWnff4Uly5ZAgsdnw13OrRi.csg_BfGVnRg37RaX3iPjNqTmsbxbd1kLM LWZL0UEGaZ2Sm6xj9_rP_F3dQtHTVc3GD.jqumDPOKZRdx2n1WEjYlfpN8uG UUjGPRYu87nKzCnHH6qTu15fR3tGsQqJ7NuikMi4pp2FDTXKPPfeNyqQHO9v 9lrVCOjJEKMyBNFNUVZ_bk6x.XUMN4..BivFdwiZgS8rX3n7whMmsNdACKoc kedm7TI.g1u7fedTth6uNzGkZiTVq.QD3ZPDxOpu3uf.2kMSkc5OLhESmfuq 0KLHke93_eZNWecL5iMNykGjH8Tr7vBnPUCD5mTL2dASKUewBFRY0ji7iNdl 7iAGLYzbY8vmQ3NS7t6IJFNcCL3Gf1DQXA_52GYuogohISiwJ8vdb8xwPqvl AzqU- X-Yahoo-SMTP: N82WFx6swBBjwcHWPFR2CGt6udzA8RPrA.xm0enFFXaK2g-- Received: from europa (mike.jeays@99.224.34.131 with login) by smtp105.rog.mail.gq1.yahoo.com with SMTP; 04 Oct 2012 19:28:04 -0700 PDT Date: Thu, 4 Oct 2012 22:28:03 -0400 From: Mike Jeays To: freebsd-questions@freebsd.org Message-ID: <20121004222803.459c4fa6@europa> In-Reply-To: <1349400979.263410106@f86.mail.ru> References: <1349400979.263410106@f86.mail.ru> X-Mailer: Claws Mail 3.7.8 (GTK+ 2.24.4; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: =?UTF-8?B?0JDQstGB0YLQuNC9INCa0LjQvA==?= Subject: Re: cksum entire dir?? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 02:30:18 -0000 On Fri, 05 Oct 2012 05:36:19 +0400 =D0=90=D0=B2=D1=81=D1=82=D0=B8=D0=BD =D0=9A=D0=B8=D0=BC wr= ote: > Hi, all, >=20 > > Paul Kraus writes: > > > > > On Tue, Sep 11, 2012 at 9:18 PM, wrote: > > > > > >> It's a real shame Unix doesn't have a really good tool for comparing > > >> two directory trees. You can use 'diff -r' (even on binaries), but t= hat > > >> fails if you have devices, named pipes, or named sockets in the > > >> filesystem. And diff or cksum don't tell you if symlinks are differe= nt. > > >> Plus you may care about file ownership, and that's where the stat > > >> command comes in handy. > > > > > > Solaris and a least a few versions of Linux have a "dircmp" command > > > that is in reality a wrapper for diff that handles special files. The > > > problem with it is that it tends to be slow (I had to validate > > > millions of files). > > > > It's not clear what the danger profile is supposed to be here; dircmp > > (and recursing 'diff' applications) can handle many cases, but mtree(8) > > (with appropriate options) covers more pathological problems. Even so, > > analysis of changes in file nodes like named sockets will usually > > require some understanding of the application. > > > > I suspect that either a recursive diff or an mtree specification is a > > good solution for the original poster's problem, but we don't have > > enough information to be more sure than that. > > > > Be well. > > Lowell >=20 > I happened to be restoring my home directory on my local machine and need= ed a way to verify that its contents were in sync with the corresponding di= rectories on a remote server. I first tried looking for an option for _rsy= nc_ that would check synchronization without actually forcibly synchronizin= g one side to the other unidirectionally, but couldn't find precisely what = I was looking for. I happened to come upon this thread, which was a coinci= dence that this same issue recently came up again. >=20 > Obviously there must be more rigorous, secure, and industrial-strength wa= ys to check synchronization between corresponding directories on remote sys= tems (apart from doing a one-way sync with _rsync_), but here's my two bits= , a quick crack at a shell function to check recursively that the contents = of two directories (and the filenames contained therein) have a high probab= ility of being in sync: >=20 > ####BEGIN CUT >=20 > # s: Function to compute recursive MD5 sum. > s ( ) { > if [ -d "$1" ] > then DIR=3D$1 > else DIR=3D. > fi > if [ `uname` =3D Linux ] > then find "$DIR" -type f -or -type l |sort |tr \\n \\0 |xargs -0 ope= nssl \ > dgst |sed s/.*\(\\\(.*\\\)\).*\ \\\(.*\\\)/\\2\ \\1/ |tee /tm= p/dgst > openssl dgst else find -s "$DIR" -type f -or -type l |tr \\n \\0 |xargs -0 md5= \ > |sed s/.*\(\\\(.*\\\)\).*\ \\\(.*\\\)/\\2\ \\1/ |tee /tm= p/dgst > md5 fi > unset DIR > rm /tmp/dgst > return > } >=20 > # sq: Function to compute recursive MD5 sum quietly. > sq ( ) { > if [ -d "$1" ] > then DIR=3D$1 > else DIR=3D. > fi > if [ `uname` =3D Linux ] > then find "$DIR" -type f -or -type l |sort |tr \\n \\0 |xargs -0 ope= nssl \ > dgst |sed s/.*\(\\\(.*\\\)\).*\ \\\(.*\\\)/\\2\ \\1/ >/tmp/dg= st > openssl dgst else find -s "$DIR" -type f -or -type l |tr \\n \\0 |xargs -0 md5= \ > |sed s/.*\(\\\(.*\\\)\).*\ \\\(.*\\\)/\\2\ \\1/ >/tmp/dg= st > md5 fi > unset DIR > rm /tmp/dgst > return > } >=20 > ####END CUT >=20 > These functions simply apply the `find ... |xargs' method suggested by pr= evious posts to output a list of MD5 digests with filenames, and then just = _md5_ the resulting file. I tried out the above in both sh(1) in FreeBSD (= my local machine) as well as in ksh(1) in Linux (the remote server), though= I haven't tested them extensively. Obviously the above are not `secure,' = and obviously an infinite number of variations are possible (such as, for e= xample, also outputting file permissions and dates of last modification wit= h ls(1) to the digest file before running _md5_ on it, to check that permis= sions and dates are also in sync). Thanks to the previous posters for solv= ing my problem! :) >=20 > All the best, > Austin "rsync --dry-run" may be a simple solution that would meet your needs? You = might need to add the "--delete" option. Take another look at man rsync.