From owner-freebsd-hubs@FreeBSD.ORG Mon Aug 17 12:59:49 2009 Return-Path: Delivered-To: freebsd-hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 33A81106568E for ; Mon, 17 Aug 2009 12:59:49 +0000 (UTC) (envelope-from Andre.Albsmeier@siemens.com) Received: from thoth.sbs.de (thoth.sbs.de [192.35.17.2]) by mx1.freebsd.org (Postfix) with ESMTP id BD1E78FC51 for ; Mon, 17 Aug 2009 12:59:48 +0000 (UTC) Received: from mail1.siemens.de (localhost [127.0.0.1]) by thoth.sbs.de (8.12.11.20060308/8.12.11) with ESMTP id n7HCYP3m006972 for ; Mon, 17 Aug 2009 14:34:25 +0200 Received: from curry.mchp.siemens.de (curry.mchp.siemens.de [139.25.40.130]) by mail1.siemens.de (8.12.11.20060308/8.12.11) with ESMTP id n7HCYPXO017642 for ; Mon, 17 Aug 2009 14:34:25 +0200 Received: (from localhost) by curry.mchp.siemens.de (8.14.3/8.14.3) id n7HCYP5S038762 for freebsd-hubs@freebsd.org; Mon, 17 Aug 2009 14:34:25 +0200 (CEST) Date: Mon, 17 Aug 2009 14:34:25 +0200 From: Andre Albsmeier To: freebsd-hubs@freebsd.org Message-ID: <20090817123425.GA15105@curry.mchp.siemens.de> References: <20090815013441.GS6527@rwpc12.mby.riverwillow.net.au> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090815013441.GS6527@rwpc12.mby.riverwillow.net.au> X-Echelon: X-Advice: Drop that crappy M$-Outlook, I'm tired of your viruses! User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: CVSup src file Edits almost alway triggering Checksum mismatch and Fixup X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2009 12:59:49 -0000 On Sat, 15-Aug-2009 at 11:34:41 +1000, John Marshall wrote: > A dialup user of our CVSup mirror alerted me to this. > > These days, almost every file updated in the src tree triggers a > Checksum mismatch, full file download and Fixup. I'm not seeing this > for doc, ports, etc.; just src. Me too ;-) The problem started (for me) a few month ago. I see the "Checksum mismatch" messages as well. Sometimes two or three, sometimes 100 or more, sometimes none. I used CVSup for years. I tried csup, nothing changed. I switched servers I CVSup from, even tried cvsup.freebsd.org. I tried compressing and non-compressing mode. Like John, I only see the problem on src, not elsewhere. Any ideas? Thanks, -Andre > > For example (from the logs of our synch's with cvsup-master)... > > --------------------------------------- > Edit ports/math/jags/Makefile,v > Edit ports/math/jags/distinfo,v > Edit src/release/Makefile,v > src/release/Makefile,v: Checksum mismatch -- will transfer entire file > Create src/release/svnbranch.awk,v > Edit src/sys/amd64/amd64/local_apic.c,v > src/sys/amd64/amd64/local_apic.c,v: Checksum mismatch -- will transfer entire file > Edit src/sys/amd64/include/apicvar.h,v > src/sys/amd64/include/apicvar.h,v: Checksum mismatch -- will transfer entire file > Edit src/sys/amd64/include/pmc_mdep.h,v > src/sys/amd64/include/pmc_mdep.h,v: Checksum mismatch -- will transfer entire file > Edit src/sys/dev/hwpmc/hwpmc_core.c,v > Edit src/sys/dev/hwpmc/hwpmc_piv.c,v > src/sys/dev/hwpmc/hwpmc_piv.c,v: Checksum mismatch -- will transfer entire file > Edit src/sys/dev/hwpmc/hwpmc_ppro.c,v > src/sys/dev/hwpmc/hwpmc_ppro.c,v: Checksum mismatch -- will transfer entire file > Edit src/sys/dev/hwpmc/hwpmc_x86.c,v > src/sys/dev/hwpmc/hwpmc_x86.c,v: Checksum mismatch -- will transfer entire file > Edit src/sys/i386/i386/local_apic.c,v > src/sys/i386/i386/local_apic.c,v: Checksum mismatch -- will transfer entire file > Edit src/sys/i386/include/apicvar.h,v > src/sys/i386/include/apicvar.h,v: Checksum mismatch -- will transfer entire file > Edit src/sys/i386/include/pmc_mdep.h,v > src/sys/i386/include/pmc_mdep.h,v: Checksum mismatch -- will transfer entire file > Edit src/sys/pci/agp_i810.c,v -> Attic > src/sys/pci/agp_i810.c,v: Checksum mismatch -- will transfer entire file > --------------------------------------- > > Followed by... > > --------------------------------------- > Applying fixups for collection cvs-all/cvs > Fixup src/release/Makefile,v > Fixup src/sys/amd64/amd64/local_apic.c,v > Fixup src/sys/amd64/include/apicvar.h,v > Fixup src/sys/amd64/include/pmc_mdep.h,v > Fixup src/sys/dev/hwpmc/hwpmc_piv.c,v > Fixup src/sys/dev/hwpmc/hwpmc_ppro.c,v > Fixup src/sys/dev/hwpmc/hwpmc_x86.c,v > Fixup src/sys/i386/i386/local_apic.c,v > Fixup src/sys/i386/include/apicvar.h,v > Fixup src/sys/i386/include/pmc_mdep.h,v > Fixup src/sys/pci/agp_i810.c,v -> Attic > --------------------------------------- > > I am not seeing Fixups for anything other than the src tree. I've > checked back through the past month's logs and the story is the same. > Is this a consequence of SVN export to CVS? Can it be fixed? Am I just > doing something wrong? > > -- > John Marshall -- Never argue with an idiot. They drag you down to their level, then beat you with their experience. From owner-freebsd-hubs@FreeBSD.ORG Tue Aug 18 18:47:36 2009 Return-Path: Delivered-To: freebsd-hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC130106568B for ; Tue, 18 Aug 2009 18:47:36 +0000 (UTC) (envelope-from simon@nitro.dk) Received: from mx.nitro.dk (zarniwoop.nitro.dk [83.92.207.38]) by mx1.freebsd.org (Postfix) with ESMTP id 863938FC64 for ; Tue, 18 Aug 2009 18:47:36 +0000 (UTC) Received: from arthur.nitro.dk (arthur.bofh [192.168.2.3]) by mx.nitro.dk (Postfix) with ESMTP id 445F42D487A for ; Tue, 18 Aug 2009 18:47:35 +0000 (UTC) Received: by arthur.nitro.dk (Postfix, from userid 1000) id 177A45C07; Tue, 18 Aug 2009 20:47:34 +0200 (CEST) Date: Tue, 18 Aug 2009 20:47:34 +0200 From: "Simon L. Nielsen" To: freebsd-hubs@freebsd.org Message-ID: <20090818184734.GA1634@arthur.nitro.dk> References: <20090815013441.GS6527@rwpc12.mby.riverwillow.net.au> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090815013441.GS6527@rwpc12.mby.riverwillow.net.au> User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: CVSup src file Edits almost alway triggering Checksum mismatch and Fixup X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 18:47:36 -0000 On 2009.08.15 11:34:41 +1000, John Marshall wrote: > These days, almost every file updated in the src tree triggers a > Checksum mismatch, full file download and Fixup. I'm not seeing this > for doc, ports, etc.; just src. > [...] > Edit src/release/Makefile,v > src/release/Makefile,v: Checksum mismatch -- will transfer entire file Hey, This is unfortunatly a known problem. The problem seems to be that the actually diff sections in the ,v files are at time re-oredered and that breaks CVSup's assumptions about the ,v files. The ,v are still valid in both before and after reorder, the order is just changed. Since cvsup's normal acelerated CVS mirror mode works by knowning how thw ,v files are layed out, cvsup then can just see that the ,v file does not look like it expects, and it does the safe thing and transfer/sync the entire file. I actually suspect if CVSup didn't do this but just transfered the delta's to the ,v files, the resulting ,v file on the client would be different, but working and valid - but this is a guess. kib@ (AFAIR) first noticed that this happened for security updates which went to multiple branches at once in the spring, but it was still only for a few files. After the upgrade of CVS to 1.12 in base (and therefor on repoman - the master CVS server) it seems to be that the reason all files are now have this problem - so CVS 1.12 probably made it worse. I suspect the reason it happens for src, but not ports or doc, is the branches. When there are no branches in a ,v file the ordering is simply newest -> oldest, but when there are branches I don't think the order is clear. So, in short I don't see any easy solutions to this, and especially given the fact that in the long run CVS/CVSup will probably be replaced by something else for distributing FreeBSD source, I don't think it will be that easy to find someone to "fix" either CVSup or CVS. PS. Just to be clear CVS/CVSup is not going away any time soon (though there are no timeline planed yet), but 3-5 years down the line I really doubt we will be using CVSup. -- Simon L. Nielsen Hats: FreeBSD.org cluster, cvs, and svn admin teams From owner-freebsd-hubs@FreeBSD.ORG Tue Aug 18 20:36:21 2009 Return-Path: Delivered-To: freebsd-hubs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 801811065693; Tue, 18 Aug 2009 20:36:21 +0000 (UTC) (envelope-from gad@FreeBSD.org) Received: from smtp7.server.rpi.edu (smtp7.server.rpi.edu [128.113.2.227]) by mx1.freebsd.org (Postfix) with ESMTP id 45DEB8FC55; Tue, 18 Aug 2009 20:36:21 +0000 (UTC) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by smtp7.server.rpi.edu (8.13.1/8.13.1) with ESMTP id n7IJUgVm016349; Tue, 18 Aug 2009 15:30:43 -0400 Mime-Version: 1.0 Message-Id: In-Reply-To: <20090818184734.GA1634@arthur.nitro.dk> References: <20090815013441.GS6527@rwpc12.mby.riverwillow.net.au> <20090818184734.GA1634@arthur.nitro.dk> Date: Tue, 18 Aug 2009 15:30:40 -0400 To: "Simon L. Nielsen" , freebsd-hubs@FreeBSD.org From: Garance A Drosehn Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Bayes-Prob: 0.0001 (Score 0) X-RPI-SA-Score: 0.10 () [Hold at 20.00] COMBINED_FROM X-CanItPRO-Stream: outgoing X-Canit-Stats-ID: Bayes signature not available X-Scanned-By: CanIt (www . roaringpenguin . com) on 128.113.2.227 Cc: Subject: Re: CVSup src file Edits almost alway triggering Checksum mismatch and Fixup X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 20:36:21 -0000 At 8:47 PM +0200 8/18/09, Simon L. Nielsen wrote: >On 2009.08.15 11:34:41 +1000, John Marshall wrote: > >> These days, almost every file updated in the src tree triggers a >> Checksum mismatch, full file download and Fixup. I'm not seeing this >> for doc, ports, etc.; just src. >> >[...] >> Edit src/release/Makefile,v >> src/release/Makefile,v: Checksum mismatch -- will transfer entire file > >Hey, > >This is unfortunatly a known problem. The problem seems to be that >the actually diff sections in the ,v files are at time re-oredered >and that breaks CVSup's assumptions about the ,v files. > >The ,v are still valid in both before and after reorder, the order >is just changed. [...] >After the upgrade of CVS to 1.12 in base (and therefor on repoman - >the master CVS server) it seems to be that the reason all files are >now have this problem - so CVS 1.12 probably made it worse. > >I suspect the reason it happens for src, but not ports or doc, is the >branches. When there are no branches in a ,v file the ordering is >simply newest -> oldest, but when there are branches I don't think >the order is clear. Hmm. I've noticed this too, and assumed that it has something to do with the SVN->CVS conversion process (which is why it would not effect ports or doc). When I've skimmed over some of my cvsup logs, I noticed it doesn't happen for all files. I've been meaning to check what is noteable about the changes where the whole file has to be transfered compared to those where it doesn't. -- Garance Alistair Drosehn = drosehn@rpi.edu Senior Systems Programmer or gad@FreeBSD.org Rensselaer Polytechnic Institute; Troy, NY; USA