From owner-svn-ports-all@freebsd.org Sun Dec 23 07:19:05 2018 Return-Path: Delivered-To: svn-ports-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1343B134EE14; Sun, 23 Dec 2018 07:19:05 +0000 (UTC) (envelope-from se@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B9BA18D4AC; Sun, 23 Dec 2018 07:19:04 +0000 (UTC) (envelope-from se@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9518998F4; Sun, 23 Dec 2018 07:19:04 +0000 (UTC) (envelope-from se@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id wBN7J41A016305; Sun, 23 Dec 2018 07:19:04 GMT (envelope-from se@FreeBSD.org) Received: (from se@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id wBN7J24l016293; Sun, 23 Dec 2018 07:19:02 GMT (envelope-from se@FreeBSD.org) Message-Id: <201812230719.wBN7J24l016293@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: se set sender to se@FreeBSD.org using -f From: Stefan Esser Date: Sun, 23 Dec 2018 07:19:02 +0000 (UTC) To: ports-committers@freebsd.org, svn-ports-all@freebsd.org, svn-ports-head@freebsd.org Subject: svn commit: r488168 - in head/misc/ctm: . files X-SVN-Group: ports-head X-SVN-Commit-Author: se X-SVN-Commit-Paths: in head/misc/ctm: . files X-SVN-Commit-Revision: 488168 X-SVN-Commit-Repository: ports MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: B9BA18D4AC X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.96)[-0.963,0]; NEURAL_HAM_LONG(-1.00)[-0.998,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-ports-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the ports tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Dec 2018 07:19:05 -0000 Author: se Date: Sun Dec 23 07:19:01 2018 New Revision: 488168 URL: https://svnweb.freebsd.org/changeset/ports/488168 Log: Apply changes developed by Stephen Montgomery-Smith and required to actually use CTM to distribute FreeBSD updates. They have been further refined by Julian H. Stacey. These changes add support for delta numbers with more than 5 digits and better compression formats. Submitted by: Stephen Montgomery-Smith, Julian H. Stacey Approved by: antoine (implicit) Added: head/misc/ctm/files/ head/misc/ctm/files/patch-ctm__rmail_ctm__rmail.8 (contents, props changed) head/misc/ctm/files/patch-ctm__rmail_ctm__rmail.c (contents, props changed) head/misc/ctm/files/patch-ctm_ctm.8 (contents, props changed) head/misc/ctm/files/patch-ctm_ctm.c (contents, props changed) head/misc/ctm/files/patch-ctm_ctm.h (contents, props changed) head/misc/ctm/files/patch-ctm_ctm__input.c (contents, props changed) head/misc/ctm/files/patch-ctm_ctm__pass1.c (contents, props changed) head/misc/ctm/files/patch-ctm_ctm__pass2.c (contents, props changed) head/misc/ctm/files/patch-ctm_ctm__pass3.c (contents, props changed) head/misc/ctm/files/patch-ctm_ctm__syntax.c (contents, props changed) head/misc/ctm/files/patch-mkCTM_mkctm.c (contents, props changed) Modified: head/misc/ctm/Makefile Modified: head/misc/ctm/Makefile ============================================================================== --- head/misc/ctm/Makefile Sun Dec 23 05:51:05 2018 (r488167) +++ head/misc/ctm/Makefile Sun Dec 23 07:19:01 2018 (r488168) @@ -2,6 +2,7 @@ PORTNAME= ctm PORTVERSION= 2.0 +PORTREVISION= 1 CATEGORIES= misc MAINTAINER= se@FreeBSD.org Added: head/misc/ctm/files/patch-ctm__rmail_ctm__rmail.8 ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/misc/ctm/files/patch-ctm__rmail_ctm__rmail.8 Sun Dec 23 07:19:01 2018 (r488168) @@ -0,0 +1,30 @@ +--- ctm_rmail/ctm_rmail.8.orig 2018-10-27 15:56:22 UTC ++++ ctm_rmail/ctm_rmail.8 +@@ -7,7 +7,7 @@ + .\" + .\" $FreeBSD$ + .\" +-.Dd January 24, 1995 ++.Dd December 23, 2018 + .Dt CTM_MAIL 8 + .Os + .Sh NAME +@@ -35,6 +35,7 @@ deltas via mail + .Op Fl p Ar piecedir + .Op Fl d Ar deltadir + .Op Fl b Ar basedir ++.Op Fl B Ar backup_dir + .Op Ar + .Sh DESCRIPTION + In conjunction with the +@@ -191,6 +192,10 @@ file in + (or if + .Li .ctm_status + does not exist). ++.It Fl B Ar backup_dir ++Specify a backup directory for use by ++.Nm ctm ++.Fl B + .It Fl D + Delete deltas after successful application by + .Xr ctm . Added: head/misc/ctm/files/patch-ctm__rmail_ctm__rmail.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/misc/ctm/files/patch-ctm__rmail_ctm__rmail.c Sun Dec 23 07:19:01 2018 (r488168) @@ -0,0 +1,30 @@ +--- ctm_rmail/ctm_rmail.c.orig 2018-12-23 07:00:45 UTC ++++ ctm_rmail/ctm_rmail.c +@@ -152,6 +152,7 @@ apply_complete() + char fname[PATH_MAX]; + char here[PATH_MAX]; + char buf[PATH_MAX*2]; ++ char *deltanamescheme[] = { "%s.%04d.gz", "%s.%04d.xz", "%s.%05d.gz", "%s.%05d.xz", NULL }; + + /* + * Grab a lock on the ctm mutex file so that we can be sure we are +@@ -200,10 +201,16 @@ apply_complete() + */ + for (;;) + { +- sprintf(delta, "%s.%04d.gz", class, ++dn); +- mk_delta_name(fname, delta); ++ ++dn; ++ for (i=0; deltanamescheme[i]; i++) ++ { ++ sprintf(delta, deltanamescheme[i], class, dn); ++ mk_delta_name(fname, delta); + +- if (stat(fname, &sb) < 0) ++ if (stat(fname, &sb) >= 0) ++ break; ++ } ++ if (!deltanamescheme[i]) + break; + + sprintf(buf, "(cd %s && ctm %s%s%s%s%s%s) 2>&1", base_dir, Added: head/misc/ctm/files/patch-ctm_ctm.8 ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/misc/ctm/files/patch-ctm_ctm.8 Sun Dec 23 07:19:01 2018 (r488168) @@ -0,0 +1,40 @@ +--- ctm/ctm.8.orig 2018-10-27 15:56:22 UTC ++++ ctm/ctm.8 +@@ -12,7 +12,7 @@ + .\" + .\" $FreeBSD$ + .\" +-.Dd April 14, 2016 ++.Dd December 23, 2018 + .Dt CTM 8 + .Os + .Sh NAME +@@ -52,8 +52,11 @@ command. + You can pass a CTM delta on stdin, or you can give the + filename as an argument. + If you do the latter, you make life a lot +-easier for your self, since the program can accept gzip'ed files and ++easier for your self, since the program can accept gzip'ed, ++bzip2'ed, or xz'ed files and + since it will not have to make a temporary copy of your file. ++(If you pass it an xz'ed file, and xz is not part of your base system, ++you will have to install xz from the ports.) + You can + specify multiple deltas at one time, they will be processed one at a + time. +@@ -272,6 +275,15 @@ contains the sequence number of the last CTM delta app + Changing + or removing this file will greatly confuse + .Nm . ++.sp ++.Pa .svn_revision ++contains the revision number emitted by SVN ++.\" eg from approx ++.\" svn up src | tail -1 | \ ++.\" sed -E 's/[^[:digit:]]//g' > src/.svn_revision ++(to reference when discussing sources with users of ++.Nm svn ++direct). + .Pp + Using the + .Fl e Added: head/misc/ctm/files/patch-ctm_ctm.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/misc/ctm/files/patch-ctm_ctm.c Sun Dec 23 07:19:01 2018 (r488168) @@ -0,0 +1,25 @@ +--- ctm/ctm.c.orig 2018-10-27 15:56:22 UTC ++++ ctm/ctm.c +@@ -213,6 +213,22 @@ Proc(char *filename, unsigned applied) + strcat(p,filename); + f = popen(p,"r"); + if(!f) { warn("%s", p); return Exit_Garbage; } ++ } else if(p && !strcmp(p,".bz2")) { ++ p = alloca(20 + strlen(filename)); ++ strcpy(p,"bzcat < "); ++ strcat(p,filename); ++ f = popen(p,"r"); ++ if(!f) { warn("%s", p); return Exit_Garbage; } ++ } else if(p && !strcmp(p,".xz")) { ++ if (system("which -s xz") != 0) { ++ fprintf(stderr, "xz is not found in $PATH. You can install it from ports, or adjust $PATH.\n"); ++ return Exit_Garbage; ++ } ++ p = alloca(20 + strlen(filename)); ++ strcpy(p,"xz -dc < "); ++ strcat(p,filename); ++ f = popen(p,"r"); ++ if(!f) { warn("%s", p); return Exit_Garbage; } + } else { + p = 0; + f = fopen(filename,"r"); Added: head/misc/ctm/files/patch-ctm_ctm.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/misc/ctm/files/patch-ctm_ctm.h Sun Dec 23 07:19:01 2018 (r488168) @@ -0,0 +1,51 @@ +--- ctm/ctm.h.orig 2018-10-27 15:56:22 UTC ++++ ctm/ctm.h +@@ -24,6 +24,7 @@ + #include + #include + #include ++#include + + #define VERSION "2.0" + +@@ -40,6 +41,8 @@ + #define CTM_F_MD5 0x05 + #define CTM_F_Count 0x06 + #define CTM_F_Bytes 0x07 ++#define CTM_F_Release 0x08 ++#define CTM_F_Forward 0x09 + + /* The qualifiers... */ + #define CTM_Q_MASK 0xff00 +@@ -47,10 +50,13 @@ + #define CTM_Q_Name_Dir 0x0200 + #define CTM_Q_Name_New 0x0400 + #define CTM_Q_Name_Subst 0x0800 ++#define CTM_Q_Name_Svnbase 0x1000 + #define CTM_Q_MD5_After 0x0100 + #define CTM_Q_MD5_Before 0x0200 + #define CTM_Q_MD5_Chunk 0x0400 + #define CTM_Q_MD5_Force 0x0800 ++#define CTM_Q_Forward_Tar 0x0100 ++#define CTM_Q_Forward_SVN 0x0200 + + struct CTM_Syntax { + char *Key; /* CTM key for operation */ +@@ -145,14 +151,16 @@ void Fatal_(int ln, char *fn, char *kind); + u_char * Ffield(FILE *fd, MD5_CTX *ctx,u_char term); + u_char * Fname(FILE *fd, MD5_CTX *ctx,u_char term,int qual, int verbose); + +-int Fbytecnt(FILE *fd, MD5_CTX *ctx, u_char term); ++intmax_t Fbytecnt(FILE *fd, MD5_CTX *ctx, u_char term); + + u_char * Fdata(FILE *fd, int u_chars, MD5_CTX *ctx); ++int Fforward(FILE *fd, intmax_t u_chars, MD5_CTX *ctx, FILE *fd_to); + + #define GETFIELD(p,q) if(!((p)=Ffield(fd,&ctx,(q)))) return BADREAD + #define GETFIELDCOPY(p,q) if(!((p)=Ffield(fd,&ctx,(q)))) return BADREAD; else p=String(p) + #define GETBYTECNT(p,q) if(0 >((p)= Fbytecnt(fd,&ctx,(q)))) return BADREAD + #define GETDATA(p,q) if(!((p) = Fdata(fd,(q),&ctx))) return BADREAD ++#define GETFORWARD(p,q) if(!Fforward(fd,(p),&ctx,q)) return BADREAD + #define GETNAMECOPY(p,q,r,v) if(!((p)=Fname(fd,&ctx,(q),(r),(v)))) return BADREAD; else p=String(p) + + int Pass1(FILE *fd, unsigned applied); Added: head/misc/ctm/files/patch-ctm_ctm__input.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/misc/ctm/files/patch-ctm_ctm__input.c Sun Dec 23 07:19:01 2018 (r488168) @@ -0,0 +1,59 @@ +--- ctm/ctm_input.c.orig 2018-10-27 15:56:22 UTC ++++ ctm/ctm_input.c +@@ -63,11 +63,11 @@ Ffield(FILE *fd, MD5_CTX *ctx,u_char term) + return buf; + } + +-int ++intmax_t + Fbytecnt(FILE *fd, MD5_CTX *ctx, u_char term) + { + u_char *p,*q; +- int u_chars=0; ++ intmax_t u_chars=0; + + p = Ffield(fd,ctx,term); + if(!p) return -1; +@@ -101,6 +101,42 @@ Fdata(FILE *fd, int u_chars, MD5_CTX *ctx) + p[u_chars] = '\0'; + return p; + } ++ ++int Fforward(FILE *fd, intmax_t u_chars, MD5_CTX *ctx, FILE *fd_to) ++{ ++ u_char buf[BUFSIZ]; ++ intmax_t amount_read = 0; ++ int amount_to_read; ++ ++ while (amount_read < u_chars) { ++ if (u_chars - amount_read >= BUFSIZ) ++ amount_to_read = BUFSIZ; ++ else ++ amount_to_read = u_chars - amount_read; ++ if(amount_to_read != fread(buf, 1, amount_to_read, fd)) { ++ Fatal("Truncated patch."); ++ return 0; ++ } ++ MD5Update(ctx,buf,amount_to_read); ++ if (fd_to != NULL) { ++ if (amount_to_read != fwrite(buf, 1, amount_to_read, fd_to)) { ++ Fatal("Write error."); ++ return 0; ++ } ++ } ++ amount_read += amount_to_read; ++ } ++ ++ if(getc(fd) != '\n') { ++ if(Verbose > 3) ++ printf("FileData wasn't followed by a newline.\n"); ++ Fatal("Corrupt patch."); ++ return 0; ++ } ++ MD5Update(ctx,"\n",1); ++ return 1; ++} ++ + + /*---------------------------------------------------------------------------*/ + /* get the filename in the next field, prepend BaseDir and give back the result Added: head/misc/ctm/files/patch-ctm_ctm__pass1.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/misc/ctm/files/patch-ctm_ctm__pass1.c Sun Dec 23 07:19:01 2018 (r488168) @@ -0,0 +1,39 @@ +--- ctm/ctm_pass1.c.orig 2018-10-27 15:56:22 UTC ++++ ctm/ctm_pass1.c +@@ -24,7 +24,8 @@ Pass1(FILE *fd, unsigned applied) + { + u_char *p,*q; + MD5_CTX ctx; +- int i,j,sep,cnt; ++ int i,j,sep; ++ intmax_t cnt, rel; + u_char *md5=0,*name=0,*trash=0; + struct CTM_Syntax *sp; + int slashwarn=0, match=0, total_matches=0; +@@ -98,7 +99,7 @@ Pass1(FILE *fd, unsigned applied) + if(Verbose > 5) + fprintf(stderr,"%s ",sp->Key); + for(i=0;(j = sp->List[i]);i++) { +- if (sp->List[i+1] && (sp->List[i+1] & CTM_F_MASK) != CTM_F_Bytes) ++ if (sp->List[i+1] && (sp->List[i+1] & CTM_F_MASK) != CTM_F_Bytes && (sp->List[i+1] & CTM_F_MASK) != CTM_F_Forward) + sep = ' '; + else + sep = '\n'; +@@ -213,6 +214,17 @@ Pass1(FILE *fd, unsigned applied) + if(md5 && strcmp(md5,p)) { + Fatal("Internal MD5 failed."); + return Exit_Garbage; ++ case CTM_F_Release: ++ GETBYTECNT(rel,sep); ++ break; ++ case CTM_F_Forward: ++ if(cnt < 0) WRONG ++ if ((j & CTM_Q_MASK) == CTM_Q_Forward_SVN && system("which -s svnadmin") != 0) { ++ fprintf(stderr, "svn is not found in $PATH. You can install it from ports/devel/subversion, or adjust $PATH.\n"); ++ return Exit_Garbage; ++ } ++ GETFORWARD(cnt,NULL); ++ break; + default: + fprintf(stderr,"List = 0x%x\n",j); + Fatal("List had garbage."); Added: head/misc/ctm/files/patch-ctm_ctm__pass2.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/misc/ctm/files/patch-ctm_ctm__pass2.c Sun Dec 23 07:19:01 2018 (r488168) @@ -0,0 +1,99 @@ +--- ctm/ctm_pass2.c.orig 2018-10-27 15:56:22 UTC ++++ ctm/ctm_pass2.c +@@ -24,7 +24,11 @@ Pass2(FILE *fd) + { + u_char *p,*q,*md5=0; + MD5_CTX ctx; +- int i,j,sep,cnt,fdesc; ++ int i,j,sep,fdesc; ++ intmax_t cnt, rel; ++ int rel2; ++ FILE *current; ++ char *current_file_name = NULL; + u_char *trash=0,*name=0; + struct CTM_Syntax *sp; + struct stat st; +@@ -32,7 +36,7 @@ Pass2(FILE *fd) + int match = 0; + char md5_1[33]; + struct CTM_Filter *filter; +- FILE *ed = NULL; ++ FILE *ed = NULL, *fd_to = NULL; + static char *template = NULL; + + if(Verbose>3) +@@ -74,7 +78,7 @@ Pass2(FILE *fd) + WRONG + found: + for(i=0;(j = sp->List[i]);i++) { +- if (sp->List[i+1] && (sp->List[i+1] & CTM_F_MASK) != CTM_F_Bytes) ++ if (sp->List[i+1] && (sp->List[i+1] & CTM_F_MASK) != CTM_F_Bytes && (sp->List[i+1] & CTM_F_MASK) != CTM_F_Forward) + sep = ' '; + else + sep = '\n'; +@@ -130,6 +134,22 @@ Pass2(FILE *fd) + sp->Key,name); + ret |= Exit_NotOK; + } ++ if (j & CTM_Q_Name_Svnbase) { ++ current_file_name = alloca(strlen(name)+128); ++ strcpy(current_file_name,name); ++ strcat(current_file_name,"/db/current"); ++ current = fopen(current_file_name,"r"); ++ if (current==NULL) { ++ fprintf(stderr,"Cannot open %s\n",current_file_name); ++ WRONG ++ } ++ if (fscanf(current,"%d",&rel2) != 1) { ++ fprintf(stderr,"Cannot find release number in %s\n",current_file_name); ++ fclose(current); ++ WRONG ++ } ++ fclose(current); ++ } + break; + } + if (j & CTM_Q_Name_File) { +@@ -285,6 +305,42 @@ Pass2(FILE *fd) + Free(p); + } + ++ break; ++ case CTM_F_Release: ++ GETBYTECNT(rel,sep); ++ if(Verbose > 3) ++ printf("Expecting release number %jd\n",rel); ++ if(Verbose > 3) ++ printf("Actual release number %d\n",rel2); ++ if (rel != rel2) { ++ fprintf(stderr,"Release number mismatch: found %d, need %jd\n",rel2,rel); ++ WRONG ++ } ++ break; ++ case CTM_F_Forward: ++ if ((j & CTM_Q_MASK) == CTM_Q_Forward_SVN) { ++ if(Verbose>3) ++ printf("This is a svn dump file and there is no certainty that it will apply cleanly.\n"); ++ GETFORWARD(cnt,NULL); ++ } ++ else if ((j & CTM_Q_MASK) == CTM_Q_Forward_Tar) { ++ if(Verbose>3) { ++ printf("This is a tar file and there is no certainty that it will apply cleanly even if it passes the following test.\n"); ++ fd_to = popen("tar tvf -","w"); ++ } else ++ fd_to = popen("tar tf - >/dev/null 2>&1","w"); ++ if (fd_to == NULL) { ++ fprintf(stderr,"Cannot forward\n"); ++ WRONG ++ } ++ GETFORWARD(cnt,fd_to); ++ if (pclose(fd_to)) { ++ fprintf(stderr,"Tar failed to close properly\n"); ++ WRONG ++ } else ++ if (Verbose > 3) ++ printf("Tar file test was good\n"); ++ } + break; + default: WRONG + } Added: head/misc/ctm/files/patch-ctm_ctm__pass3.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/misc/ctm/files/patch-ctm_ctm__pass3.c Sun Dec 23 07:19:01 2018 (r488168) @@ -0,0 +1,170 @@ +--- ctm/ctm_pass3.c.orig 2018-10-27 15:56:22 UTC ++++ ctm/ctm_pass3.c +@@ -35,10 +35,12 @@ Pass3(FILE *fd) + { + u_char *p,*q,buf[BUFSIZ]; + MD5_CTX ctx; +- int i,j,sep,cnt; ++ int i,j,sep; ++ intmax_t cnt,rel; ++ char *svn_command = NULL; + u_char *md5=0,*md5before=0,*trash=0,*name=0,*uid=0,*gid=0,*mode=0; + struct CTM_Syntax *sp; +- FILE *ed=0; ++ FILE *ed=0, *fd_to; + struct stat st; + char md5_1[33]; + int match=0; +@@ -131,7 +133,7 @@ Pass3(FILE *fd) + WRONG + found: + for(i=0;(j = sp->List[i]);i++) { +- if (sp->List[i+1] && (sp->List[i+1] & CTM_F_MASK) != CTM_F_Bytes) ++ if (sp->List[i+1] && (sp->List[i+1] & CTM_F_MASK) != CTM_F_Bytes && (sp->List[i+1] & CTM_F_MASK) != CTM_F_Forward) + sep = ' '; + else + sep = '\n'; +@@ -149,53 +151,98 @@ Pass3(FILE *fd) + break; + case CTM_F_Count: GETBYTECNT(cnt,sep); break; + case CTM_F_Bytes: GETDATA(trash,cnt); break; ++ case CTM_F_Release: GETBYTECNT(rel,sep); break; ++ case CTM_F_Forward: ++ if ((j & CTM_Q_MASK) == CTM_Q_Forward_Tar) { ++ if (Verbose > 0) ++ fd_to = popen("tar xvf -","w"); ++ else ++ fd_to = popen("tar xvf - >/dev/null 2>&1","w"); ++ } else if ((j & CTM_Q_MASK) == CTM_Q_Forward_SVN) { ++ svn_command = alloca(strlen(name)+128); ++ if (Verbose > 0) ++ snprintf(svn_command,strlen(name)+127,"svnadmin load %s\n", name); ++ else ++ snprintf(svn_command,strlen(name)+127,"svnadmin load %s > /dev/null 2>&1\n", name); ++ fd_to = popen(svn_command,"w"); ++ } else WRONG ++ if (fd_to == NULL) { ++ fprintf(stderr,"Cannot forward\n"); ++ WRONG ++ } ++ if (Verbose > 0) { ++ if (!strcmp(sp->Key,"TR")) ++ fprintf(stderr,"> %s\n",sp->Key); ++ else ++ fprintf(stderr,"> %s %s\n",sp->Key,name); ++ } ++ GETFORWARD(cnt,fd_to); ++ if (pclose(fd_to)) { ++ if ((j & CTM_Q_MASK) == CTM_Q_Forward_Tar) ++ fprintf(stderr,"Tar failed to close properly\n"); ++ else if ((j & CTM_Q_MASK) == CTM_Q_Forward_SVN) ++ fprintf(stderr,"Svnadmin failed to close properly\n"); ++ WRONG ++ } ++ if ((j & CTM_Q_MASK) == CTM_Q_Forward_SVN) { ++ snprintf(svn_command,strlen(name)+127,"svnadmin pack %s\n", name); ++ if (system(svn_command)) { ++ fprintf(stderr,"\"%s\" didn't work.", svn_command); ++ WRONG ++ } ++ } ++ break; + default: WRONG + } + } +- /* XXX This should go away. Disallow trailing '/' */ +- j = strlen(name)-1; +- if(name[j] == '/') name[j] = '\0'; + +- /* +- * If a filter list is specified, run thru the filter list and +- * match `name' against filters. If the name matches, set the +- * required action to that specified in the filter. +- * The default action if no filterlist is given is to match +- * everything. +- */ ++ if (name) { ++ /* XXX This should go away. Disallow trailing '/' */ ++ j = strlen(name)-1; ++ if(name[j] == '/') name[j] = '\0'; + +- match = (FilterList ? !(FilterList->Action) : CTM_FILTER_ENABLE); +- for (filter = FilterList; filter; filter = filter->Next) { +- if (0 == regexec(&filter->CompiledRegex, name, +- 0, 0, 0)) { +- match = filter->Action; ++ /* ++ * If a filter list is specified, run thru the filter list and ++ * match `name' against filters. If the name matches, set the ++ * required action to that specified in the filter. ++ * The default action if no filterlist is given is to match ++ * everything. ++ */ ++ ++ match = (FilterList ? !(FilterList->Action) : CTM_FILTER_ENABLE); ++ for (filter = FilterList; filter; filter = filter->Next) { ++ if (0 == regexec(&filter->CompiledRegex, name, ++ 0, 0, 0)) { ++ match = filter->Action; ++ } + } +- } + +- if (CTM_FILTER_DISABLE == match) /* skip file if disabled */ +- continue; ++ if (CTM_FILTER_DISABLE == match) /* skip file if disabled */ ++ continue; + +- if (Verbose > 0) ++ if (Verbose > 0 && strcmp(sp->Key,"SV") && strcmp(sp->Key,"TR")) + fprintf(stderr,"> %s %s\n",sp->Key,name); +- if(!strcmp(sp->Key,"FM") || !strcmp(sp->Key, "FS")) { +- i = open(name,O_WRONLY|O_CREAT|O_TRUNC,0666); +- if(i < 0) { +- warn("%s", name); +- WRONG ++ if(!strcmp(sp->Key,"FM") || !strcmp(sp->Key, "FS")) { ++ i = open(name,O_WRONLY|O_CREAT|O_TRUNC,0666); ++ if(i < 0) { ++ warn("%s", name); ++ WRONG ++ } ++ if(cnt != write(i,trash,cnt)) { ++ warn("%s", name); ++ WRONG ++ } ++ close(i); ++ if(strcmp(md5,MD5File(name,md5_1))) { ++ fprintf(stderr," %s %s MD5 didn't come out right\n", ++ sp->Key,name); ++ WRONG ++ } ++ if (settime(name,times)) WRONG ++ continue; + } +- if(cnt != write(i,trash,cnt)) { +- warn("%s", name); +- WRONG +- } +- close(i); +- if(strcmp(md5,MD5File(name,md5_1))) { +- fprintf(stderr," %s %s MD5 didn't come out right\n", +- sp->Key,name); +- WRONG +- } +- if (settime(name,times)) WRONG +- continue; + } ++ + if(!strcmp(sp->Key,"FE")) { + ed = popen("ed","w"); + if(!ed) { +@@ -278,6 +325,8 @@ Pass3(FILE *fd) + } + continue; + } ++ if(!strcmp(sp->Key,"TR") || !strcmp(sp->Key,"SV")) ++ continue; + WRONG + } + Added: head/misc/ctm/files/patch-ctm_ctm__syntax.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/misc/ctm/files/patch-ctm_ctm__syntax.c Sun Dec 23 07:19:01 2018 (r488168) @@ -0,0 +1,44 @@ +--- ctm/ctm_syntax.c.orig 2018-10-27 15:56:22 UTC ++++ ctm/ctm_syntax.c +@@ -22,16 +22,21 @@ + #define MD5 CTM_F_MD5 + #define Count CTM_F_Count + #define Bytes CTM_F_Bytes ++#define Release CTM_F_Release ++#define Forward CTM_F_Forward + + /* The qualifiers... */ + #define File CTM_Q_Name_File + #define Dir CTM_Q_Name_Dir ++#define Svnbase CTM_Q_Name_Svnbase + #define New CTM_Q_Name_New + #define Subst CTM_Q_Name_Subst + #define After CTM_Q_MD5_After + #define Before CTM_Q_MD5_Before + #define Chunk CTM_Q_MD5_Chunk + #define Force CTM_Q_MD5_Force ++#define Tar CTM_Q_Forward_Tar ++#define SVN CTM_Q_Forward_SVN + + static int ctmFM[] = /* File Make */ + { Name|File|New|Subst, Uid, Gid, Mode, +@@ -57,6 +62,12 @@ static int ctmDM[] = /* Directory Make */ + static int ctmDR[] = /* Directory Remove */ + { Name|Dir, 0 }; + ++static int ctmTR[] = /* Forward to tar */ ++ { Count, Forward|Tar, 0 }; ++ ++static int ctmSV[] = /* Forward to svnadmin load */ ++ { Name|Dir|Svnbase, Release, Count, Forward|SVN, 0 }; ++ + struct CTM_Syntax Syntax[] = { + { "FM", ctmFM }, + { "FS", ctmFS }, +@@ -66,4 +77,6 @@ struct CTM_Syntax Syntax[] = { + { "AS", ctmAS }, + { "DM", ctmDM }, + { "DR", ctmDR }, ++ { "TR", ctmTR }, ++ { "SV", ctmSV }, + { 0, 0} }; Added: head/misc/ctm/files/patch-mkCTM_mkctm.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/misc/ctm/files/patch-mkCTM_mkctm.c Sun Dec 23 07:19:01 2018 (r488168) @@ -0,0 +1,269 @@ +--- mkCTM/mkctm.c.orig 2018-10-27 15:56:22 UTC ++++ mkCTM/mkctm.c +@@ -181,12 +181,16 @@ Equ(const char *dir1, const char *dir2, const char *na + goto finish; + } + #endif +- p1=mmap(0, s1.st_size, PROT_READ, MAP_PRIVATE, fd1, 0); +- if (p1 == (u_char *)MAP_FAILED) { err(3, "%s", buf1); } ++ if (s1.st_size) { ++ p1=mmap(0, s1.st_size, PROT_READ, MAP_PRIVATE, fd1, 0); ++ if (p1 == (u_char *)MAP_FAILED) { err(3, "%s", buf1); } ++ } + close(fd1); + +- p2=mmap(0, s2.st_size, PROT_READ, MAP_PRIVATE, fd2, 0); +- if (p2 == (u_char *)MAP_FAILED) { err(3, "%s", buf2); } ++ if (s2.st_size) { ++ p2=mmap(0, s2.st_size, PROT_READ, MAP_PRIVATE, fd2, 0); ++ if (p2 == (u_char *)MAP_FAILED) { err(3, "%s", buf2); } ++ } + close(fd2); + + /* If identical, we're done. */ +@@ -222,6 +226,9 @@ Equ(const char *dir1, const char *dir2, const char *na + int j; + FILE *F; + ++ if (!s1.st_size || !s2.st_size) ++ goto subst; ++ + if (s1.st_size && p1[s1.st_size-1] != '\n') { + if (verbose > 0) + fprintf(stderr, +@@ -295,8 +302,10 @@ Equ(const char *dir1, const char *dir2, const char *na + free(ob); + } + finish: +- munmap(p1, s1.st_size); +- munmap(p2, s2.st_size); ++ if (s1.st_size) ++ munmap(p1, s1.st_size); ++ if (s2.st_size) ++ munmap(p2, s2.st_size); + } + } + +@@ -325,15 +334,19 @@ Add(const char *dir1, const char *dir2, const char *na + fd1 = open(buf2, O_RDONLY); + if (fd1 < 0) { err(3, "%s", buf2); } + fstat(fd1, &st); +- p1=mmap(0, st.st_size, PROT_READ, MAP_PRIVATE, fd1, 0); +- if (p1 == (u_char *)MAP_FAILED) { err(3, "%s", buf2); } ++ if (st.st_size) { ++ p1=mmap(0, st.st_size, PROT_READ, MAP_PRIVATE, fd1, 0); ++ if (p1 == (u_char *)MAP_FAILED) { err(3, "%s", buf2); } ++ } + close(fd1); + m2 = MD5Data(p1, st.st_size, md5_2); + name_stat("CTMFM", dir2, name, de); + printf(" %s %u\n", m2, (unsigned)st.st_size); +- fwrite(p1, 1, st.st_size, stdout); ++ if (st.st_size) ++ fwrite(p1, 1, st.st_size, stdout); + putchar('\n'); +- munmap(p1, st.st_size); ++ if (st.st_size) ++ munmap(p1, st.st_size); + s_new_files++; + s_new_bytes += st.st_size; + } +@@ -493,6 +506,172 @@ DoDir(const char *dir1, const char *dir2, const char * + free(nl2); + } + ++void ++SvnAdd(const char *dir1, const char *dir2, struct dirent *de) ++{ ++ char current_file[] = "/db/current"; ++ char *buf2 = alloca(strlen(dir2) + strlen(current_file) + strlen(de->d_name) + 4); ++ char *tmpdir = getenv("TMPDIR"); ++ if (tmpdir == NULL) ++ tmpdir = strdup(_PATH_TMP); ++ char tmpfilebase[] = "/CTMserver.XXXXXXXXXX"; ++ char *tmpfilename = alloca(strlen(tmpdir)+strlen(tmpfilebase)+4); ++ int command_size = strlen(dir2) + strlen(tmpdir)+strlen(tmpfilebase) + strlen(de->d_name) + 128; ++ char *command = alloca(command_size+1); ++ int ret_val; ++ ++ strcpy(buf2, dir2); strcat(buf2, "/"); strcat(buf2, de->d_name); strcat(buf2, current_file); ++ strcpy(tmpfilename, tmpdir); strcat(tmpfilename, tmpfilebase); ++ mktemp(tmpfilename); ++ ++ snprintf(command,command_size,"tar -C %s -cvf %s %s 2>&%d\n",dir2,tmpfilename,de->d_name,fileno(logf)); ++ fflush(logf); ++ ret_val = system(command); ++ if (ret_val!=0) errx(1,"The command \"%s\" failed with return value %d",command,ret_val); ++ printf("CTMTR "); ++ change += 2; /* Make sure change is big enough .*/ ++ ++ StatFile(tmpfilename); ++ printf("%jd\n", st.st_size); ++ snprintf(command,command_size,"cat %s; rm -f %s\n",tmpfilename,tmpfilename); ++ ret_val = system(command); ++ if (ret_val!=0) errx(1,"The command \"%s\" failed with return value %d",command,ret_val); ++ putchar('\n'); ++} ++ ++void ++SvnEqu(const char *dir1, const char *dir2, struct dirent *de) ++{ ++ char current_file[] = "/db/current"; ++ char *buf1 = alloca(strlen(dir1) + strlen(current_file) + strlen(de->d_name) + 4); ++ char *buf2 = alloca(strlen(dir2) + strlen(current_file) + strlen(de->d_name) + 4); ++ char *tmpdir = getenv("TMPDIR"); ++ if (tmpdir == NULL) ++ tmpdir = strdup(_PATH_TMP); ++ char tmpfilebase[] = "/CTMserver.XXXXXXXXXX"; ++ char *tmpfilename = alloca(strlen(tmpdir)+strlen(tmpfilebase)+4); ++ int command_size = strlen(dir2) + strlen(tmpdir)+strlen(tmpfilebase) + strlen(de->d_name) + 128; ++ char *command = alloca(command_size+1); ++ long int release1, release2; ++ FILE *current1, *current2; ++ int ret_val; ++ ++ strcpy(buf1, dir1); strcat(buf1, "/"); strcat(buf1, de->d_name); strcat(buf1, current_file); ++ strcpy(buf2, dir2); strcat(buf2, "/"); strcat(buf2, de->d_name); strcat(buf2, current_file); ++ strcpy(tmpfilename, tmpdir); strcat(tmpfilename, tmpfilebase); ++ mktemp(tmpfilename); ++ ++ current1 = fopen(buf1,"r"); ++ current2 = fopen(buf2,"r"); ++ ++ if (current1 != NULL) { ++ fscanf(current1,"%ld",&release1); ++ fclose(current1); ++ } else ++ errx(1,"No db/release in %s",buf1); ++ if (current2 != NULL) { ++ fscanf(current2,"%ld",&release2); ++ fclose(current2); ++ } else ++ errx(1,"No db/release in %s",buf2); ++ ++ if (release2 > release1) { ++ snprintf(command,command_size,"svnadmin dump %s/%s -r %ld:%ld --incremental --deltas 2>&%d > %s\n",dir2,de->d_name,release1+1,release2,fileno(logf),tmpfilename); ++ fflush(logf); ++ ret_val = system(command); ++ if (ret_val!=0) errx(1,"The command \"%s\" failed with return value %d",command,ret_val); ++ printf("CTMSV %s %ld ", de->d_name, release1); ++ change += 2; /* Make sure change is big enough .*/ ++ ++ StatFile(tmpfilename); ++ printf("%jd\n", st.st_size); ++ snprintf(command,command_size,"cat %s; rm -f %s\n",tmpfilename,tmpfilename); ++ ret_val = system(command); ++ if (ret_val!=0) errx(1,"The command \"%s\" failed with return value %d",command,ret_val); ++ putchar('\n'); ++ } ++} ++ ++void ++DoSvn(const char *dir1, const char *dir2) ++{ ++ int i1, i2, n1, n2, i; ++ struct dirent **nl1, **nl2; ++ char *buf1 = alloca(strlen(dir1) + 4); ++ char *buf2 = alloca(strlen(dir2) + 4); ++ ++ strcpy(buf1, dir1); strcat(buf1, "/"); ++ strcpy(buf2, dir2); strcat(buf2, "/"); ++ n1 = scandir(buf1, &nl1, dirselect, alphasort); ++ n2 = scandir(buf2, &nl2, dirselect, alphasort); ++ i1 = i2 = -1; ++ GetNext(&i1, &n1, nl1, dir1, "", &s1_ignored, &s1_bogus, &s1_wrong); ++ GetNext(&i2, &n2, nl2, dir2, "", &s2_ignored, &s2_bogus, &s2_wrong); ++ for (;i1 < n1 || i2 < n2;) { ++ ++ if (damage_limit && damage > damage_limit) ++ break; ++ ++ /* Get next item from list 1 */ ++ if (i1 < n1 && !nl1[i1]) ++ GetNext(&i1, &n1, nl1, dir1, "", ++ &s1_ignored, &s1_bogus, &s1_wrong); ++ ++ /* Get next item from list 2 */ ++ if (i2 < n2 && !nl2[i2]) ++ GetNext(&i2, &n2, nl2, dir2, "", ++ &s2_ignored, &s2_bogus, &s2_wrong); ++ ++ if (i1 >= n1 && i2 >= n2) { ++ /* Done */ ++ break; ++ } else if (i1 >= n1 && i2 < n2) { ++ /* end of list 1, add anything left on list 2 */ ++ if (nl2[i2]->d_type == DT_REG) { ++ if (strcmp(nl2[i2]->d_name,".ctm_status")==0) ++ Add(dir1, dir2, "", nl2[i2]); ++ else ++ errx(1,"Improper file found in svn archive"); ++ } else ++ SvnAdd(dir1, dir2, nl2[i2]); ++ free(nl2[i2]); nl2[i2] = 0; ++ } else if (i1 < n1 && i2 >= n2) { ++ /* end of list 2, delete anything left on list 1 */ ++ Del(dir1, dir2, "", nl1[i1]); ++ free(nl1[i1]); nl1[i1] = 0; ++ } else if (!(i = strcmp(nl1[i1]->d_name, nl2[i2]->d_name))) { ++ /* Identical names */ ++ if (nl2[i1]->d_type == DT_REG && nl2[i2]->d_type == DT_REG && strcmp(nl2[i2]->d_name,".ctm_status")==0) ++ Equ(dir1, dir2, "", nl1[i1]); ++ else if (nl2[i1]->d_type == DT_DIR && nl2[i2]->d_type == DT_DIR) ++ SvnEqu(dir1, dir2, nl1[i1]); ++ else ++ ++ errx(1,"Improper file found in svn archive"); ++ free(nl1[i1]); nl1[i1] = 0; ++ free(nl2[i2]); nl2[i2] = 0; ++ } else if (i < 0) { ++ /* Something extra in list 1, delete it */ ++ Del(dir1, dir2, "", nl1[i1]); ++ free(nl1[i1]); nl1[i1] = 0; ++ } else { ++ /* Something extra in list 2, add it */ ++ if (nl2[i2]->d_type == DT_REG) { ++ if (strcmp(nl2[i2]->d_name,".ctm_status")==0) ++ Add(dir1, dir2, "", nl2[i2]); ++ else ++ errx(1,"Improper file found in svn archive"); ++ } else ++ SvnAdd(dir1, dir2, nl2[i2]); ++ free(nl2[i2]); nl2[i2] = 0; ++ } ++ } ++ if (n1 >= 0) ++ free(nl1); ++ if (n2 >= 0) ++ free(nl2); ++} ++ + int + main(int argc, char **argv) + { +@@ -581,17 +760,22 @@ main(int argc, char **argv) + argv[0], argv[1], argv[2], argv[3]); + printf("CTM_BEGIN 2.0 %s %s %s %s\n", + argv[0], argv[1], argv[2], argv[3]); +- DoDir(argv[4], argv[5], ""); ++ if (strncmp(argv[0],"svn",3) == 0) ++ DoSvn(argv[4], argv[5]); ++ else ++ DoDir(argv[4], argv[5], ""); + if (damage_limit && damage > damage_limit) { + print_stat(stderr, "DAMAGE: "); + errx(1, "damage of %d would exceed %d files", + damage, damage_limit); +- } else if (change < 2) { ++/* change <= 2 means no change because of .ctm_status and .svn_revision */ ++ } else if (change < 3) { + errx(4, "no changes"); + } else { + printf("CTM_END "); + fprintf(logf, "CTM_END\n"); +- print_stat(stderr, "END: "); ++ if (strncmp(argv[0],"svn",3) != 0) ++ print_stat(stderr, "END: "); + } + exit(0); + }