Date: Thu, 30 Jan 1997 14:49:56 -0800 (PST) From: Simon Shapiro <Shimon@i-Connect.Net> To: Tom Samplonius <tom@sdf.com> Cc: hackers@freebsd.org Subject: Re: More 2.2-BETA goodies... Message-ID: <XFMail.970130154550.Shimon@i-Connect.Net> In-Reply-To: <Pine.NEB.3.94.970129035230.18719A-100000@misery.sdf.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --]
Hi Tom Samplonius; On 29-Jan-97 you wrote:
>
> On Wed, 29 Jan 1997, Simon Shapiro wrote:
>
> ...
> > If there is interest, I can post it. It runs on Slowlaris 2.5.1, linux
2.x
> > and FreeBSD 9of courcse). We prefer it to others because it does true
> ...
>
> I'm interested. I've been looking for something to test transactional
> limits of SCSI controllers and drives. Problems with tagged command
> queuing, etc don't show up until you have a very high number of
> transactions per second.
>
> On an other note, have you tried turning AHC_TAGENABLE and/or
> AHC_SCBPAGING_ENABLE on? AHC_TAGENABLE is pretty reliable if your drives
> support it. SCSPAGING needs some work yet, I think.
Thanx. I am sure many will be interested. I am saving this note myself.
We are working, as many of you have learned by now :-) on writing a FreeBSD
driver for the DPT controllers as we need some unique features in these
(expensive) controllers. I say ``expensive'' as our initial work will only
apply to the high-end adapters.
Here is st.c, AKA rs.c. (st was known as rs for many, many years. I found
/usr/local/bin/rs upon installing FreeBSD. It does something totally
different.
If this program has any apeal, I can contribute it to the FreeBSD project.
How? I know to change the copyright message...
Anyway, no man page for now, but basically, you give it a filename to
work on, a size is optional. and what kind of a mess you want it to create.
A cute addision is that it finds the size of raw partitions very quickly
(stat(2) does not) and -S will tell you what it found.
It does reads by default, but can do read-modify-write or write-only.
Random seek is the default behavior, but you can cause it to seek
sequentially.
It tries to report ``net'' times. I found that different Unixes report
times(2) very differently. Slowlaris, for example, appears to discount
(part of?) ``blocked-on-i/o'' time.
It misses calendar ``wall clock'' time but ``date;st -....;date'' does the
same.
There is a million options. see the source and the -h option.
It needs no makefile. Just compile it.
Simon
[-- Attachment #2 --]
/******************************************************************************
* *
* Copyright (c) 1990-1995 by Simon shapiro *
* All Rights Reserved *
* *
* THIS IS UNPUBLISHED PROPRIETARY SOURCE CODE OF *
* Simon shapiro *
* *
* The copyright notice above does not evidence any *
* actual or intended publication of such source code. *
* *
******************************************************************************/
/******************************************************************************
* *
* rs: Random and Sequential Seek read test program *
* *
* arguments: -f file_name *
* -p no_of_passes (default = 0) *
* -r record_size (default = 4096) *
* -s file_size (default = 0 & means auto-detect) *
* -R no_records (default = file_size / record_size) *
* -v File system is very, very fast (RAM disk) *
* -l Use lockf(2) to lock the records. Do NOT sleep on busy *
* -L Use lockf but sleep if busy *
* -o Use O_SYNC/fsync to guarantee synchronous writes *
* -O records to synchronize records every so often (fsync only) *
* -q Do sequential access instead of random *
* -Q Do not explicitly lseek in sequential mode *
* -W Enable writes (-w will not operate without this ``safety'' *
* -w R|W:pattern Write options: r means read then write, *
* w:pattern means fill the buffer *
* with pattern (hex no.)*
* -x debug mode on (default = off) *
* -S Report file size (handy for raw devices) *
* -m Report time in milliseconds *
* -M Report result in megabytes/second Mutex with -I *
* -I Report number of I/O operations per second *
* -V Give visual progress indication (dot per record) *
* *
* returns: 0 and no of clock ticks passed *
* *
* caveats: One pass per call *
* Wierd record sizes will result in sub-optimal results on most *
* systems (wierd is where record_size % NBPSC (512) != 0) *
* If file_size % record_size != 0 we make it so (truncate) *
* If the file does not exist, or too short we fail! *
* We do not check for holes in files *
* We pass through (ignore) symbolic links *
* Running on non-random-seeking character special can make life *
* exciting :-) *
* Makes file_size/record_size reads per pass *
* if very_fast (-v) is on, we time the whole itteration, not *
* just the read portion *
* In choosing format, Mb/sec has higher priority than msc *
* If mb_per_sec, then record size must be a fraction of 1Mb *
* If Mb/Sec, do not let file_size * passes (total bytes read) *
* excceed 2GB (0x7fffffff) or reported results will be funny *
* Both a valid -w (see below) and -W must exist to enable writes.*
* -w w:pattern assumes that pattern is a ulong. *
* Write options do NOT work with -Q (skip seeking). *
* Random seek mode (default) does not guarantee that all records *
* will be accessed. There will be as many accesses as there are *
* blocks in the file. *
* *
******************************************************************************/
#ident "$Header: /usr/src/local/rs/RCS/st.c,v 1.5 1997/01/14 00:54:25 ShimonR Exp ShimonR $"
#include <sys/types.h>
#ifdef __FreeBSD__
#include <limits.h>
#endif
#include <values.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <sys/times.h>
#include <fcntl.h>
#include <time.h>
#include <limits.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>
#ifdef __FreeBSD__
typedef unsigned long ulong;
#define F_LOCK LOCK_SH
#define F_TLOCK LOCK_NB
#define F_ULOCK LOCK_UN
#define O_SYNC 0
#include <sys/file.h>
#endif
#ifndef OFF_MAX
#define OFF_MAX LONG_MAX
#endif
#define DO_WRITES 0x11
#define VERSION "Random Seeker Version $Revision: 1.5 $\n"
#define VALID_OPTIONS "O:R:oQqVSMmvxf:r:s:p:Ww:IlL"
#define USAGE "\
Usage: rs -f file_name [options...]\n\
Options: -p no_of_passes (default = 1)\n\
-r record_size (default = 4096)\n\
-s file_size (default = 0 & means auto-detect)\n\
-R no_records (default = file_size / record_size)\n\
-q Perform seQential access, instead of random\n\
-Q Do not explicitly seek in sequential mode\n\
-v File system is very, very fast (RAM disk)\n\
-l Use non-blocking lockf(2)/flock(2)\n\
-L Use blocking lockf(2)/flock()\n\
-o Force synchronized writes (Linux only?)\n\
-W Enable writes\n\
-w R|W:pattern Do read-then-write or write, hex pattern\n\
-S Report file size (handy for raw devices)\n\
-m Report time in milliseconds\n\
-M|I Report result in megabytes or I/O-ops sper /second\n\
-x debug mode on (default = off)\n\
-V Give visual indication of progres\n"
#define STAT_FAILED_MSG "rs: Failed to obtain status of %s"
#define NOT_VALID "NV"
#define CANNOT_OPEN_DEV "rs: Cannot OPEN %s (R/O) for sizing"
#define CANNOT_SEEK_DEV "rs: Cannot SEEK %s for sizing"
#define RECORD_SIZE 4096
#define PASSES 1
/* Exit status values */
#ifndef BAD_OPTION
#define BAD_OPTION 1
#endif
#define STAT_FAILED 2
#define BAD_FILE_TYPE 3
#define MALLOC_FAILED 4
#define OPEN_FAILED 5
#define LSEEK_FAILED 6
#define READ_FAILED 7
#define CLOSE_FAILED 8
#define MEGABYTE (1024 * 1024)
static int is_disk(mode_t mode_word)
{
switch ( mode_word & S_IFMT )
{
case S_IFCHR:
case S_IFBLK:
return(1);
default:
return(0);
}
}
static int invalid_file(mode_t mode_word)
{
switch ( mode_word & S_IFMT )
{
case S_IFIFO:
/* case S_IFCHR: */
case S_IFDIR:
return(1);
default:
return(0);
}
}
static int seek_endpoint(int fd, size_t offset)
{
char result[1];
int read_result;
if ( lseek(fd, offset, SEEK_SET) == -1 )
return(-1);
read_result = read(fd, (void *)result, 1);
switch ( read_result )
{
case -1:
return(-2);
case 1:
return(0);
default:
return(1);
}
}
int
main(argc, argv)
int argc;
char **argv;
{
/* External functions */
extern char *optarg;
/* Local Variables */
int bad_option = 0,
ndx,
passes = PASSES,
device_fd = -1,
debug_on = 0,
very_fast = 0,
milliseconds = 0,
mb_per_sec = 0,
io_per_sec = 0,
tell_file_size = 0,
visual_progress = 0,
sequential_access = 0,
do_lseek = 1,
do_lockf = 0,
blocking_lock = 0,
do_writes = 0,
read_then_write = 0,
enable_writes = 0,
do_o_sync = 0,
recs_per_fsync = 0;
ulong write_pattern = 0;
size_t record_size = RECORD_SIZE,
file_size = 0,
records,
no_of_reads = 0;
char *file_name = NULL,
*buffer;
struct tms times_struct;
clock_t start_time = 0,
finish_time = 0,
accumulated_time = 0;
struct stat status_buffer;
/* Parse command line */
if ( argc == 1 )
++bad_option;
else
{
while ( (ndx = getopt(argc, argv, VALID_OPTIONS)) != -1)
{
switch (ndx)
{
case 'o':
++do_o_sync;
break;
case 'O':
recs_per_fsync = atoi(optarg);
break;
case 'W':
enable_writes = 0x01;
break;
case 'w':
switch ( optarg[0] )
{
case 'R':
++read_then_write;
do_writes = 1;
break;
case 'W':
if ( optarg[1] == ':' )
{
write_pattern = optarg[2];
do_writes = 0x10;
}
else
{
do_writes = 0;
++bad_option;
}
break;
default:
++bad_option;
break;
}
break;
case 'Q':
do_lseek = 0;
break;
case 'q':
++sequential_access;
break;
case 'V':
++visual_progress;
break;
case 'S':
++tell_file_size;
break;
case 'M':
++mb_per_sec;
io_per_sec = 0;
break;
case 'l':
++do_lockf;
blocking_lock = 0;
break;
case 'L':
++do_lockf;
++blocking_lock;
break;
case 'I':
++io_per_sec;
mb_per_sec = 0;
break;
case 'm':
++milliseconds;
break;
case 'v':
++very_fast;
break;
case 'x':
++debug_on;
break;
case 'f':
file_name = optarg;
if ( (file_name == NULL) || !file_name )
++bad_option;
break;
case 'R':
no_of_reads = (int) strtoul(optarg, (char **)NULL, 0);
if ( !no_of_reads )
++bad_option;
break;
case 's':
file_size = (int) strtoul(optarg, (char **)NULL, 0);
if ( !file_size )
++bad_option;
break;
case 'r':
record_size = (int) strtoul(optarg, (char **)NULL, 0);
if ( !record_size )
++bad_option;
break;
case 'p':
passes = (int) strtoul(optarg, (char **)NULL, 0);
if ( passes < 1 )
++bad_option;
break;
case '?':
++bad_option;
break;
}
if ( bad_option )
break;
}
}
if ( recs_per_fsync && ! do_o_sync )
{
(void)fprintf(stderr,
"%s Ooops: For syncronous WRITE operations, specify both -o and -O!\n",
argv[0]);
exit(BAD_OPTION);
}
/* Verify (carefully!) that we relly want to write anything */
if ( (do_writes && ! enable_writes) || (enable_writes && !do_writes) )
{
(void)fprintf(stderr,
"%s Ooops: For WRITE operations, specify both -W and -w!\n",
argv[0]);
exit(BAD_OPTION);
}
else
do_writes |= enable_writes;
if ( !enable_writes )
read_then_write = 0;
/* lseek can be disabled only in sequential READ-only mode */
if ( !sequential_access )
++do_lseek;
if ( enable_writes && !do_lseek )
{
(void) fprintf(stderr, "%s cannot accept -W and -Q together!\n", argv[0]);
exit(BAD_OPTION);
}
if ( debug_on )
(void) fprintf(stderr, VERSION);
if ( bad_option ||
!file_name ||
!record_size ||
( (file_size!= 0) && ( file_size < record_size))
)
{
(void) fprintf(stdout, USAGE);
exit(BAD_OPTION);
}
if ( stat(file_name, &status_buffer) )
{
perror("Failed to obtain status of requested file!");
exit ( STAT_FAILED );
}
if ( invalid_file(status_buffer.st_mode) )
{
perror("Cannot perform random seeks on this type of a file!");
exit ( BAD_FILE_TYPE );
}
/* if file size is not known yet, find it out */
if ( !file_size )
{
if ( debug_on )
(void)fprintf(stderr, "[%s:%d]: File_size is unknown...\n",
__FILE__, __LINE__);
if ( is_disk(status_buffer.st_mode) )
{
if ( debug_on )
(void)fprintf(stderr, "[%s:%d]: Appears to be a disk...\n",
__FILE__, __LINE__);
/* there is no known Unix system call to report size of a device...
* So we will open it and find (the hard way ) its size
*/
if ( (device_fd = open(file_name, O_RDONLY)) == -1 )
{
perror("Failed to open requested file");
exit( OPEN_FAILED );
}
else
{
/* For lack of better way of finding a device size,
* do a binary search...
* set top mark
* set bottom mark
* try half way.
* if failed top mark = half way
* else bottom = half way.
* repeat until top - bottom <= 1
*/
register size_t top_size = LONG_MAX,
bottom_size = 0,
half_size = 0;
ndx = 0;
do
{
half_size = bottom_size + (top_size - bottom_size) / 2;
switch ( seek_endpoint(device_fd, half_size) )
{
case -1:
perror("lseek(2) failed while searching file size!");
return( LSEEK_FAILED );
case -2:
case 1:
top_size = half_size;
break;
case 0:
bottom_size = half_size;
break;
}
++ndx;
}
while ( (top_size - bottom_size) > 1 );
file_size = (size_t)top_size;
if ( debug_on )
(void)fprintf(stderr,
"[%s:%d]: Disk size of %ul determined in %d passes\n",
__FILE__, __LINE__, file_size, ndx);
}
(void) close(device_fd);
device_fd = -1;
}
else
{
file_size = (size_t)status_buffer.st_size;
if ( debug_on )
(void)fprintf(stderr,
"[%s:%d]: O/S reported disk size of %u\n",
__FILE__, __LINE__, file_size);
}
}
if ( tell_file_size )
{
if ( mb_per_sec )
{
(void) fprintf(stdout, "Actual Size = %u.%02uMB, ",
file_size / (1024 * 1024),
(file_size % (1024 * 1024) / 10000));
}
else
(void) fprintf(stdout, "Actual Size = %u, ", file_size);
}
/* Round file size down to complete number of records */
file_size = record_size * ( file_size / record_size);
if ( tell_file_size )
{
if ( mb_per_sec )
(void) fprintf(stdout, "Used Size = %u.%02uMB\n",
file_size / MEGABYTE,
(file_size % MEGABYTE / 10000));
else
(void) fprintf(stdout, "Used Size = %u\n", file_size);
}
if ( debug_on )
(void)fprintf(stderr, "[%s:%d]: File_size is %u\n",
__FILE__, __LINE__, file_size);
/* Allocate memory for the input buffer */
if ( (buffer = malloc(record_size)) == NULL )
{
perror("Cannot allocate buffer for read operations");
exit ( MALLOC_FAILED );
}
if ( debug_on )
(void)fprintf(stderr,
"[%s:%d]: Allocated %d bytes of buffer space\n",
__FILE__, __LINE__, record_size);
/* there is a libc thing that does that. What's its name? */
if ( do_writes == DO_WRITES )
{
(void)memset((void *)buffer, write_pattern, record_size);
}
/* If necessary, open the file */
if ( device_fd == -1 )
{
int mode;
if ( enable_writes )
{
mode = O_RDWR;
if ( do_o_sync )
{
mode |= O_SYNC;
}
}
else
{
mode = O_RDONLY;
}
if ( (device_fd = open(file_name, mode)) == -1 )
{
perror("Failed to open requested file");
return( OPEN_FAILED );
}
if ( debug_on )
(void)fprintf(stderr,
"[%s:%d]: Opened %s\n",
__FILE__, __LINE__, file_name);
}
/* We set this one late, to let all other computation be done first */
records = file_size / record_size;
if ( !no_of_reads )
{
no_of_reads = records;
}
/* loop as much as necessary */
for (ndx = 0; ndx < passes; ndx++ )
{
register int read_no = 0,
fsync_no = 0;
/* Initialize the random number generator, once per iteration */
if ( !sequential_access )
srand((long)time((time_t *)0));
if ( debug_on )
(void)fprintf(stderr,
"[%s:%d]: Pass %d\n",
__FILE__, __LINE__, ndx);
if ( visual_progress )
(void) fprintf(stdout, "\n%d: ", ndx);
if ( very_fast )
start_time = times(×_struct);
/* Lseek initially if no seeks wanted */
if ( !do_lseek )
{
if ( lseek(device_fd, 0, SEEK_SET) == -1 )
perror("Failed to pre-seek");
if ( debug_on )
(void)fprintf(stderr, "[%s:%d]: Pre-seeked to record 0\n",
__FILE__, __LINE__);
}
for ( read_no = 0; read_no < no_of_reads; read_no++ )
{
register size_t record_no = 0;
register size_t offset = 0;
register ssize_t read_result = 0,
write_result = 0;
if ( sequential_access )
{
if ( do_lseek )
record_no = read_no;
}
else
{
ssize_t random = lrand48();
record_no = random % records;
if ( debug_on )
(void)fprintf(stderr,
"[%s.%d]: random = %u, reads = %u, record_no = %u\n",
__FILE__, __LINE__, random, records, record_no);
}
if ( do_lseek )
{
offset = record_no * record_size;
if ( lseek(device_fd, offset, SEEK_SET) == -1 )
{
perror(sequential_access ? "Failed to seek in Sequential mode"
: "Failed to seek in Random mode");
exit (LSEEK_FAILED );
}
if ( debug_on )
(void)fprintf(stderr,
"[%s:%d]: Access %d seeked to record %u of %u\n",
__FILE__, __LINE__, read_no, record_no, no_of_reads);
else
if ( visual_progress )
(void) fputc('.', stdout);
}
if ( do_lockf )
{
int lock_mode;
if ( blocking_lock )
lock_mode = F_LOCK;
else
lock_mode = F_TLOCK;
#ifdef __FreeBSD__
if ( flock(device_fd, lock_mode) == -1 )
#else
if ( lockf(device_fd, lock_mode, record_size) == -1 )
#endif
{
#ifdef __FreeBSD__
if ( (errno == EWOULDBLOCK) && (lock_mode == F_TLOCK) )
#else
if ( (errno == EAGAIN) && (lock_mode == F_TLOCK) )
#endif
{
if ( debug_on )
{
(void)fprintf(stderr,
"[%s:%d]: Record %d is already locked\n",
__FILE__, __LINE__, record_no);
}
else
{
if ( visual_progress )
(void) fputc('L', stdout);
else
{
(void) fprintf(stderr,
"%s ERROR: Failed to lock record %d (%d)\n",
argv[0], record_no, errno);
exit(1);
}
}
}
}
}
if ( visual_progress )
(void) fputc('.', stdout);
/* Take a time stamp.
* We do not check return value as it can only fail if the
* pointer is invalid...
*/
if ( !very_fast )
start_time = times(×_struct);
if ( do_writes == DO_WRITES )
{
write_result = write(device_fd, (void *)buffer, record_size);
if ( write_result != record_size)
{
(write_result == (MAXINT))
? perror("Write operation failed: ")
: (void)fprintf(stderr, "%s Short Write!\n", argv[0]);
if ( do_lockf )
#ifdef __FreeBSD__
(void) flock(device_fd, F_ULOCK);
#else
(void) lockf(device_fd, F_ULOCK, record_size);
#endif
goto abrupt_end;
}
else
{
if ( recs_per_fsync )
{
if ( ++fsync_no == recs_per_fsync )
{
fsync_no = 0;
if ( fsync(device_fd) )
{
perror("Failed to synchronize writes");
goto abrupt_end;
}
}
}
}
}
else
{
read_result = (ssize_t)read(device_fd, (void *)buffer, record_size);
if ( read_result != record_size)
{
if (read_result == -1)
perror("Read operation failed: ");
else (void)fprintf(stderr, "%s Short Read (%d)!\n", argv[0],
read_result);
goto abrupt_end;
}
if ( read_then_write )
{
if ( lseek(device_fd, offset, SEEK_SET) == -1 )
{
perror(sequential_access ? "Failed to re-seek in Sequential mode"
: "Failed to re-seek in Random mode");
exit (LSEEK_FAILED );
}
write_result = write(device_fd, (void *)buffer, record_size);
if ( write_result != record_size)
{
(write_result == -1)
? perror("Re-Write operation failed: ")
: (void)fprintf(stderr, "%s Short Re-Write (%d)!\n",
argv[0], write_result);
goto abrupt_end;
}
}
}
#ifdef __FreeBSD__
(void)flock(device_fd, F_ULOCK);
#else
(void)lockf(device_fd, F_ULOCK, record_size);
#endif
if ( !very_fast )
finish_time = times(×_struct);
if ( !very_fast )
accumulated_time += (finish_time - start_time);
if ( !very_fast && debug_on )
(void)fprintf(stderr,
"[%s:%d]: Done in %lu ticks, total is %lu ticks\n",
__FILE__, __LINE__,
finish_time - start_time, accumulated_time);
}
if ( very_fast )
{
finish_time = times(×_struct);
accumulated_time += (finish_time - start_time);
}
}
abrupt_end:
if ( accumulated_time == 0 )
exit(1);
if (visual_progress )
(void) fputc('\n', stdout);
if ( mb_per_sec )
{
double bytes_per_tic = (double)file_size * (double)passes / (double)accumulated_time;
(void) fprintf(stdout, "%.2f\n", bytes_per_tic);
}
else if ( io_per_sec )
{
double result = (double)CLK_TCK * (double)file_size * (double)passes \
/ (double)record_size / (double)accumulated_time;
(void) fprintf(stdout, "%.2f\n",result);
}
else if ( milliseconds )
{
double result = (double)accumulated_time * (double)1000 / (double)CLK_TCK;
(void) fprintf(stdout, "%.2f\n", result);
}
else
{
(void) fprintf(stdout, "%lu\n", accumulated_time);
}
return(0);
}
help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?XFMail.970130154550.Shimon>