Date: Wed, 20 Mar 1996 09:21:35 -0500 (EST) From: "Ron G. Minnich" <rminnich@Sarnoff.COM> To: hackers@freebsd.org Subject: new software available: vx, vector execute program for many machines Message-ID: <Pine.SOL.3.91.960320090557.18864A-100000@morse>
next in thread | raw e-mail | index | archive | help
VX. VX stands for Vector eXecute. VX can start processes up on lots of remote machines very quickly. We have measured times on a cluster of 48 machines of 320 milliseconds to run /bin/date (wall-clock); in a 100+ machine environment we could run /bin/date in 10 seconds, wall-clock time. In the 100+ machine environment the dominant overhead was YP lookup; using the built-in name to address mapping the time would be shorter. This code includes a library to quickly manage hosts lists, but so far (sadly) I only have the man page for VX. History: original program (mrex, for Multi REX) based on Vector Kernel RPC and slightly more secure rexd built in late 1992; mrex was changed and renamed hrex by Gene Kim in may 1993. VX is a TCP-based version which communicates to standard exec daemons. You have to supply a user name and password when you run programs, e.g.: time vx p0,p1,p5 /bin/date Name (rminnich): Password (rminnich): Wed Mar 20 09:13:22 EST 1996 Wed Mar 20 09:19:00 EST 1996 Wed Mar 20 09:17:51 EST 1996 0.019u 0.029s 0:02.35 1.2% 104+345k 0+0io 0pf+0w Note that most of the runtime was to let me type my password ... You can use regular expressions to specify machines. The command below says to use the r.e. 'any host with p as the first letter': tres 27% time vx -lH \^p /bin/date Name (rminnich): Password (rminnich): p0 : Wed Mar 20 09:15:43 EST 1996 p1 : Wed Mar 20 09:21:21 EST 1996 p9 : Wed Mar 20 14:19:43 EST 1996 p5 : Wed Mar 20 09:20:12 EST 1996 p13: Wed Mar 20 14:24:31 EST 1996 p11: Wed Mar 20 14:16:17 EST 1996 p6 : Wed Mar 20 09:20:57 EST 1996 p8 : Wed Mar 20 14:12:12 EST 1996 p10: Wed Mar 20 14:16:31 EST 1996 p14: Wed Mar 20 14:15:56 EST 1996 p2 : Wed Mar 20 09:16:48 EST 1996 p15: Wed Mar 20 14:18:14 EST 1996 p4 : Wed Mar 20 09:20:24 EST 1996 p12: Wed Mar 20 14:16:17 EST 1996 p7 : Wed Mar 20 09:23:54 EST 1996 p3 : Wed Mar 20 09:15:50 EST 1996 0.033u 0.033s 0:02.94 2.0% 55+210k 0+0io 0pf+0w note some things: 1) i used the -l switch so the machine names would be printed. 2) you can use lists of re's as well as a list of hosts and a list of re's. 3) the time for 16 machines was almost the same as the time for 3. In fact, this is pretty much the time we've seen for 48 machines. It's also much faster than anything else we've tried for large numbers of machines. For 16 machines or fewer you can put together files full of rsh commands such as: rsh -n p0 /bin/date & and it is not much slower, but it's slower. It's also much less flexible. Anyway, source, man page for vx, and source to the hostlists library is included. Bugs to me, of course. I've been using this stuff in one form or another to support cluster computing for a few years. In fact, the hostslists stuff was used in a compiler runtime. I am pretty sure it is pretty solid. You'll find it at the URL below. Note it is an FTP, NOT HTTP, URL. ron Ron Minnich |" Microsoft Word: It does so little and it does rminnich@sarnoff.com | it so slowly" -- Maya Gokhale (609)-734-3120 | ftp://ftp.sarnoff.com/pub/mnfs/www/docs/cluster.html
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.SOL.3.91.960320090557.18864A-100000>