From owner-freebsd-current@FreeBSD.ORG Mon Dec 22 05:05:47 2014 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 34286DB2 for ; Mon, 22 Dec 2014 05:05:47 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E81CF3C75 for ; Mon, 22 Dec 2014 05:05:46 +0000 (UTC) Received: from [73.34.117.227] (helo=ilsoft.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1Y2vBh-000OLt-7M; Mon, 22 Dec 2014 05:05:45 +0000 Received: from revolution.hippie.lan (revolution.hippie.lan [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id sBM55hn2022401; Sun, 21 Dec 2014 22:05:43 -0700 (MST) (envelope-from ian@freebsd.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 73.34.117.227 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1/BPpASurELn9Fpcr7DcYSE Message-ID: <1419224743.1018.108.camel@freebsd.org> Subject: Re: simple task to speed up booting From: Ian Lepore To: Poul-Henning Kamp Date: Sun, 21 Dec 2014 22:05:43 -0700 In-Reply-To: <43445.1418553160@critter.freebsd.dk> References: <43445.1418553160@critter.freebsd.dk> Content-Type: multipart/mixed; boundary="=-rYYRf5HywhHjvQvdsgPg" X-Mailer: Evolution 3.12.8 FreeBSD GNOME Team Port Mime-Version: 1.0 Cc: current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Dec 2014 05:05:47 -0000 --=-rYYRf5HywhHjvQvdsgPg Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Sun, 2014-12-14 at 10:32 +0000, Poul-Henning Kamp wrote: > The rotating swirlie ('-/|\') in the loader accounts for a surprisingly > large part of our boot time on systems with slow-ish serial consoles. > > I think right now it takes a step for each 512 byte read, reducing that > to once every 64kB or even 1MB would be an improvement with the kind of > kernel sizes we have today. > I investigated this a bit today. I instrumented the loader on arm to count how many times twiddle() is called while loading a 5.5MB kernel. When loading over NFS it was called 5580 times. When loading from an sdcard it was called 284 times. Poking around in the code, it looks like NFS calls twiddle() once per 1K block read, CD9660 once per 2K block, and UFS and ext2fs once per filesystem blocksize block. All of them have some other scattered twiddle calls while navigating metadata that probably don't add up to much compared to the bulk reading. My amd64 kernel is twice as big even with lots of the generic devices removed; that's a lot of IO. But one twiddle per filesystem block shouldn't be so bad... let's call it 600 twiddles, each one writes 2 bytes to serial, so a total of ~1200mS at 9600bps. The same kernel loaded over nfs would cost over 20 seconds in serial output. So all in all it seems like different kinds of IO need different throttling, something like the attached (which also still has some stats output in it). I can't decide if it's worth committing... it'll have a lot of value to someone with slow serial and netbooting, is that common? -- Ian --=-rYYRf5HywhHjvQvdsgPg Content-Description: Content-Disposition: inline; filename="twiddle_divisors.diff" Content-Type: text/x-patch; name="twiddle_divisors.diff"; charset="us-ascii" Content-Transfer-Encoding: 7bit Index: lib/libstand/cd9660.c =================================================================== --- lib/libstand/cd9660.c (revision 275941) +++ lib/libstand/cd9660.c (working copy) @@ -281,7 +281,7 @@ cd9660_open(const char *path, struct open_file *f) buf = malloc(buf_size = ISO_DEFAULT_BLOCK_SIZE); vd = buf; for (bno = 16;; bno++) { - twiddle(); + twiddle(1); rc = f->f_dev->dv_strategy(f->f_devdata, F_READ, cdb2devb(bno), ISO_DEFAULT_BLOCK_SIZE, buf, &read); if (rc) @@ -314,7 +314,7 @@ cd9660_open(const char *path, struct open_file *f) while (off < dsize) { if ((off % ISO_DEFAULT_BLOCK_SIZE) == 0) { - twiddle(); + twiddle(1); rc = f->f_dev->dv_strategy (f->f_devdata, F_READ, cdb2devb(bno + boff), @@ -374,7 +374,7 @@ cd9660_open(const char *path, struct open_file *f) /* Check for Rock Ridge since we didn't in the loop above. */ bno = isonum_733(rec.extent) + isonum_711(rec.ext_attr_length); - twiddle(); + twiddle(1); rc = f->f_dev->dv_strategy(f->f_devdata, F_READ, cdb2devb(bno), ISO_DEFAULT_BLOCK_SIZE, buf, &read); if (rc) @@ -431,7 +431,7 @@ buf_read_file(struct open_file *f, char **buf_p, s if (fp->f_buf == (char *)0) fp->f_buf = malloc(ISO_DEFAULT_BLOCK_SIZE); - twiddle(); + twiddle(32); rc = f->f_dev->dv_strategy(f->f_devdata, F_READ, cdb2devb(blkno), ISO_DEFAULT_BLOCK_SIZE, fp->f_buf, &read); if (rc) Index: lib/libstand/ext2fs.c =================================================================== --- lib/libstand/ext2fs.c (revision 275941) +++ lib/libstand/ext2fs.c (working copy) @@ -353,7 +353,7 @@ ext2fs_open(const char *upath, struct open_file *f /* allocate space and read super block */ fs = (struct ext2fs *)malloc(sizeof(*fs)); fp->f_fs = fs; - twiddle(); + twiddle(1); error = (f->f_dev->dv_strategy)(f->f_devdata, F_READ, EXT2_SBLOCK, EXT2_SBSIZE, (char *)fs, &buf_size); if (error) @@ -395,7 +395,7 @@ ext2fs_open(const char *upath, struct open_file *f len = blkgrps * fs->fs_bsize; fp->f_bg = malloc(len); - twiddle(); + twiddle(1); error = (f->f_dev->dv_strategy)(f->f_devdata, F_READ, EXT2_SBLOCK + EXT2_SBSIZE / DEV_BSIZE, len, (char *)fp->f_bg, &buf_size); @@ -507,7 +507,7 @@ ext2fs_open(const char *upath, struct open_file *f if (error) goto out; - twiddle(); + twiddle(1); error = (f->f_dev->dv_strategy)(f->f_devdata, F_READ, fsb_to_db(fs, disk_block), fs->fs_bsize, buf, &buf_size); @@ -568,7 +568,7 @@ read_inode(ino_t inumber, struct open_file *f) * Read inode and save it. */ buf = malloc(fs->fs_bsize); - twiddle(); + twiddle(1); error = (f->f_dev->dv_strategy)(f->f_devdata, F_READ, ino_to_db(fs, fp->f_bg, inumber), fs->fs_bsize, buf, &rsize); if (error) @@ -665,7 +665,7 @@ block_map(struct open_file *f, daddr_t file_block, if (fp->f_blk[level] == (char *)0) fp->f_blk[level] = malloc(fs->fs_bsize); - twiddle(); + twiddle(1); error = (f->f_dev->dv_strategy)(f->f_devdata, F_READ, fsb_to_db(fp->f_fs, ind_block_num), fs->fs_bsize, fp->f_blk[level], &fp->f_blksize[level]); @@ -723,7 +723,7 @@ buf_read_file(struct open_file *f, char **buf_p, s bzero(fp->f_buf, block_size); fp->f_buf_size = block_size; } else { - twiddle(); + twiddle(4); error = (f->f_dev->dv_strategy)(f->f_devdata, F_READ, fsb_to_db(fs, disk_block), block_size, fp->f_buf, &fp->f_buf_size); Index: lib/libstand/nandfs.c =================================================================== --- lib/libstand/nandfs.c (revision 275941) +++ lib/libstand/nandfs.c (working copy) @@ -921,7 +921,7 @@ nandfs_bmap_lookup(struct nandfs *fs, struct nandf return (0); } - twiddle(); + twiddle(1); NANDFS_DEBUG("calling get_map with %jx\n", ind_block_num); map = nandfs_get_map(fs, node, ind_block_num, phys); if (map == NULL) Index: lib/libstand/nfs.c =================================================================== --- lib/libstand/nfs.c (revision 275941) +++ lib/libstand/nfs.c (working copy) @@ -662,7 +662,7 @@ nfs_read(struct open_file *f, void *buf, size_t si (int)fp->off); #endif while ((int)size > 0) { - twiddle(); + twiddle(32); cc = nfs_readdata(fp, fp->off, (void *)addr, size); /* XXX maybe should retry on certain errors */ if (cc == -1) { @@ -1311,7 +1311,7 @@ nfs_read(struct open_file *f, void *buf, size_t si (int)fp->off); #endif while ((int)size > 0) { - twiddle(); + twiddle(32); cc = nfs_readdata(fp, fp->off, (void *)addr, size); /* XXX maybe should retry on certain errors */ if (cc == -1) { Index: lib/libstand/read.c =================================================================== --- lib/libstand/read.c (revision 275941) +++ lib/libstand/read.c (working copy) @@ -77,7 +77,7 @@ read(int fd, void *dest, size_t bcount) return (-1); } if (f->f_flags & F_RAW) { - twiddle(); + twiddle(4); errno = (f->f_dev->dv_strategy)(f->f_devdata, F_READ, btodb(f->f_offset), bcount, dest, &resid); if (errno) Index: lib/libstand/stand.h =================================================================== --- lib/libstand/stand.h (revision 275941) +++ lib/libstand/stand.h (working copy) @@ -242,7 +242,8 @@ extern int sprintf(char *buf, const char *cfmt, .. extern int snprintf(char *buf, size_t size, const char *cfmt, ...) __printflike(3, 4); extern void vsprintf(char *buf, const char *cfmt, __va_list); -extern void twiddle(void); +extern void twiddle(u_int divisor); +extern void twiddle_stat(void); extern void ngets(char *, int); #define gets(x) ngets((x), 0) Index: lib/libstand/tftp.c =================================================================== --- lib/libstand/tftp.c (revision 275941) +++ lib/libstand/tftp.c (working copy) @@ -453,8 +453,7 @@ tftp_read(struct open_file *f, void *addr, size_t while (size > 0) { int needblock, count; - if (!(tc++ % 16)) - twiddle(); + twiddle(32); needblock = tftpfile->off / tftpfile->tftp_blksize + 1; Index: lib/libstand/twiddle.c =================================================================== --- lib/libstand/twiddle.c (revision 275941) +++ lib/libstand/twiddle.c (working copy) @@ -42,11 +42,20 @@ __FBSDID("$FreeBSD$"); /* Extra functions from NetBSD standalone printf.c */ +static u_int twidcount; +static u_int twiddone; + +void twiddle_stat() {printf("twiddle count %u done %u\n", twidcount, twiddone);} + void -twiddle() +twiddle(u_int divisor) { static int pos; + twidcount++; + if (divisor > 1 && (twidcount % divisor) != 0) + return; + twiddone++; putchar("|/-\\"[pos++ & 3]); putchar('\b'); } Index: lib/libstand/ufs.c =================================================================== --- lib/libstand/ufs.c (revision 275941) +++ lib/libstand/ufs.c (working copy) @@ -155,7 +155,7 @@ read_inode(inumber, f) * Read inode and save it. */ buf = malloc(fs->fs_bsize); - twiddle(); + twiddle(1); rc = (f->f_dev->dv_strategy)(f->f_devdata, F_READ, fsbtodb(fs, ino_to_fsba(fs, inumber)), fs->fs_bsize, buf, &rsize); @@ -265,7 +265,7 @@ block_map(f, file_block, disk_block_p) if (fp->f_blk[level] == (char *)0) fp->f_blk[level] = malloc(fs->fs_bsize); - twiddle(); + twiddle(1); rc = (f->f_dev->dv_strategy)(f->f_devdata, F_READ, fsbtodb(fp->f_fs, ind_block_num), fs->fs_bsize, @@ -346,7 +346,7 @@ buf_write_file(f, buf_p, size_p) if (fp->f_buf == (char *)0) fp->f_buf = malloc(fs->fs_bsize); - twiddle(); + twiddle(4); rc = (f->f_dev->dv_strategy)(f->f_devdata, F_READ, fsbtodb(fs, disk_block), block_size, fp->f_buf, &fp->f_buf_size); @@ -365,7 +365,7 @@ buf_write_file(f, buf_p, size_p) * Write the block out to storage. */ - twiddle(); + twiddle(4); rc = (f->f_dev->dv_strategy)(f->f_devdata, F_WRITE, fsbtodb(fs, disk_block), block_size, fp->f_buf, &fp->f_buf_size); @@ -406,7 +406,7 @@ buf_read_file(f, buf_p, size_p) bzero(fp->f_buf, block_size); fp->f_buf_size = block_size; } else { - twiddle(); + twiddle(4); rc = (f->f_dev->dv_strategy)(f->f_devdata, F_READ, fsbtodb(fs, disk_block), block_size, fp->f_buf, &fp->f_buf_size); @@ -515,7 +515,7 @@ ufs_open(upath, f) /* allocate space and read super block */ fs = malloc(SBLOCKSIZE); fp->f_fs = fs; - twiddle(); + twiddle(1); /* * Try reading the superblock in each of its possible locations. */ @@ -649,7 +649,7 @@ ufs_open(upath, f) if (rc) goto out; - twiddle(); + twiddle(1); rc = (f->f_dev->dv_strategy)(f->f_devdata, F_READ, fsbtodb(fs, disk_block), fs->fs_bsize, buf, &buf_size); Index: lib/libstand/write.c =================================================================== --- lib/libstand/write.c (revision 275941) +++ lib/libstand/write.c (working copy) @@ -80,7 +80,7 @@ write(fd, dest, bcount) return (-1); } if (f->f_flags & F_RAW) { - twiddle(); + twiddle(4); errno = (f->f_dev->dv_strategy)(f->f_devdata, F_WRITE, btodb(f->f_offset), bcount, dest, &resid); if (errno) Index: sys/boot/common/boot.c =================================================================== --- sys/boot/common/boot.c (revision 275941) +++ sys/boot/common/boot.c (working copy) @@ -189,6 +189,8 @@ autoboot(int timeout, char *prompt) } } + twiddle_stat(); + if (timeout >= 0) { otime = time(NULL); when = otime + timeout; /* when to boot */ --=-rYYRf5HywhHjvQvdsgPg--