Date: Tue, 31 Oct 2000 14:24:15 +0000 (GMT) From: George Reid <greid@ukug.uk.freebsd.org> To: FreeBSD-gnats-submit@freebsd.org Subject: bin/22442: [PATCH] Increase speed of split(1) Message-ID: <Pine.BSF.4.21.0010311419040.4159-100000@sobek.nevernet.net>
next in thread | raw e-mail | index | archive | help
>Number: 22442
>Category: bin
>Synopsis: [PATCH] Increase speed of split(1)
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: change-request
>Submitter-Id: current-users
>Arrival-Date: Tue Oct 31 06:30:01 PST 2000
>Closed-Date:
>Last-Modified:
>Originator: George Reid
>Release: FreeBSD 5.0-CURRENT i386
>Organization:
FreeBSD UK User Group
>Environment:
>Description:
The speed of split(1) can be increased by using dynamically-allocated
buffers.
The following results are from a large file created with
dd if=/dev/zero of=bigfile bs=1024k count=300
$ /usr/bin/time split -b 1024k bigfile
90.19 real 0.02 user 22.42 sys
$ /usr/bin/time ./mysplit -b 1024k bigfile
81.81 real 0.04 user 25.60 sys
This could represent problems on systems with low memory (perhaps further
modification is needed), but given that the average amount of memory being
distributed with PCs nowadays, this might be useful.
>How-To-Repeat:
>Fix:
--- split.c.orig Tue Oct 31 14:11:46 2000
+++ split.c Tue Oct 31 14:16:30 2000
@@ -62,7 +62,7 @@
long numlines; /* Line count to split on. */
int file_open; /* If a file open. */
int ifd = -1, ofd = -1; /* Input/output file descriptors. */
-char bfr[MAXBSIZE]; /* I/O buffer. */
+char *bfr; /* I/O buffer. */
char fname[MAXPATHLEN]; /* File name prefix. */
regex_t rgx;
int pflag;
@@ -176,11 +176,16 @@
int dist, len;
char *C;
+ if((bfr = (char *)malloc(bytecnt)) == NULL)
+ err(EX_OSERR, "malloc");
+
for (bcnt = 0;;)
- switch ((len = read(ifd, bfr, MAXBSIZE))) {
+ switch ((len = read(ifd, bfr, bytecnt))) {
case 0:
+ free(bfr);
exit(0);
case -1:
+ free(bfr);
err(EX_IOERR, "read");
/* NOTREACHED */
default:
@@ -195,22 +200,29 @@
len -= bytecnt, C += bytecnt) {
newfile();
if (write(ofd,
- C, (int)bytecnt) != bytecnt)
+ C, (int)bytecnt) != bytecnt) {
+ free(bfr);
err(EX_IOERR, "write");
+ }
}
if (len != 0) {
newfile();
- if (write(ofd, C, len) != len)
+ if (write(ofd, C, len) != len) {
+ free(bfr);
err(EX_IOERR, "write");
+ }
} else
file_open = 0;
bcnt = len;
} else {
bcnt += len;
- if (write(ofd, bfr, len) != len)
+ if (write(ofd, bfr, len) != len) {
+ free(bfr);
err(EX_IOERR, "write");
+ }
}
}
+ free(bfr);
}
/*
@@ -227,6 +239,9 @@
if ((infp = fdopen(ifd, "r")) == NULL)
err(EX_NOINPUT, "fdopen");
+ if((bfr = (char *)malloc(MAXBSIZE)) == NULL)
+ err(EX_OSERR, "malloc");
+
/* Process input one line at a time */
while (fgets(bfr, sizeof(bfr), infp) != NULL) {
const int len = strlen(bfr);
@@ -254,9 +269,13 @@
newfile();
/* Write out line */
- if (write(ofd, bfr, len) != len)
+ if (write(ofd, bfr, len) != len) {
+ free(bfr);
err(EX_IOERR, "write");
+ }
}
+
+ free(bfr);
/* EOF or error? */
if (ferror(infp))
>Release-Note:
>Audit-Trail:
>Unformatted:
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-bugs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.21.0010311419040.4159-100000>