From owner-freebsd-current@FreeBSD.ORG Fri Jul 25 04:43:17 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D0E9FA3A; Fri, 25 Jul 2014 04:43:17 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 978732FD7; Fri, 25 Jul 2014 04:43:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=Ie+AdQPg5R1l/u9a7a+Pxk+x7qwupq/UOFfWXLW5iZI=; b=Xx7CAFPA3VJCpY45vOlypNGfQRkQp8CkgofxoWDHQvjVEsHjIu79PQw2l/564cSq4j6V9+q3a7UfA5XX4a49bm9d2uXv/a4PeYc7BnRIyHi52nbpP/cdQ/iAW+fKRi+Zb3No4ISe6YRp71SwNkhnwXO10w7MYqiPwndKZv2yrtM=; Received: from localhost.lerctr.org ([127.0.0.1]:10110 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XAXLa-000PSa-Qu; Thu, 24 Jul 2014 23:43:14 -0500 Received: from 107-128-180-255.lightspeed.austtx.sbcglobal.net ([107.128.180.255]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 24 Jul 2014 23:43:10 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 24 Jul 2014 23:43:10 -0500 From: Larry Rosenman To: Allan Jude , Mark Martinec Subject: Re: zfs send/recv: STILL invalid Backup Stream In-Reply-To: <53D1AB3D.1060205@freebsd.org> References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> <53D15438.6040105@freebsd.org> <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> <9eef3e7e964cb33fd163cc2f9300f326@mailbox.ijs.si> <53D1AB3D.1060205@freebsd.org> Message-ID: <33c478c3120c21d6cb9c325cc1119cf8@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: Freebsd fs , freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2014 04:43:18 -0000 On 2014-07-24 19:56, Allan Jude wrote: > On 2014-07-24 20:46, Mark Martinec wrote: >> 2014-07-25 01:36 Larry Rosenman wrote: >> >>> #!/bin/sh >>> DATE=`date "+%Y-%m-%d"` >>> #DATE2=2013-03-24 >>> #DATE2=`date -v "-1d" "+%Y-%m-%d"` >>> # snap the source >>> ssh root@tbh.lerctr.org zfs snapshot -r zroot@${DATE} >>> # zfs copy the source to here. >>> ssh root@tbh.lerctr.org "zfs send -v -R zroot@${DATE} | \ >>> ssh home.lerctr.org \"zfs recv -F -u -v -d zroot/backups/TBH2\"" >> >> Btw, this double-ssh looks awkward, why not just: >> >> ssh root@tbh.lerctr.org "zfs send ..." | zfs recv ... >> >> or better yet: >> >> ssh root@tbh.lerctr.org "zfs send ..." | mbuffer -m 16M | zfs recv >> ... >> >> (The misc/mbuffer compensates for bursty zfs reads and writes. >> A note to myself: I should suggest to Allan to add mbuffer >> in a pipe as used in sysutils/zxfer, instead of patching zxfer >> for our local use :) >> >> Mark >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to >> "freebsd-current-unsubscribe@freebsd.org" > > zxfer can already do this, with the -D option > I actually use misc/clpbar and get a progress bar as well > > -D 'bar -s %%size%% -bl 1m -bs 128m' > > or in your case: -D 'mbuffer -m 16M' Ok, I did the mbuffer trick, and the SEND side is where the memory issue is: borg.lerctr.org /home/ler/bin $ tail zfs-send.log 23:28:12 15.7G zroot/home@2014-07-24_22:56 23:28:13 15.7G zroot/home@2014-07-24_22:56 23:28:14 15.7G zroot/home@2014-07-24_22:56 23:28:15 15.7G zroot/home@2014-07-24_22:56 23:28:16 15.7G zroot/home@2014-07-24_22:56 23:28:17 15.7G zroot/home@2014-07-24_22:56 23:28:18 15.7G zroot/home@2014-07-24_22:56 23:28:19 15.7G zroot/home@2014-07-24_22:56 23:28:20 15.8G zroot/home@2014-07-24_22:56 Write failed: Cannot allocate memory borg.lerctr.org /home/ler/bin $ borg.lerctr.org /home/ler/bin $ tail zfs-recv.log cannot receive new filesystem stream: invalid backup stream borg.lerctr.org /home/ler/bin $ borg.lerctr.org /home/ler/bin $ cat backup-TBH-ZFS-initial.sh #!/bin/sh DATE=`date "+%Y-%m-%d_%H:%M"` #DATE2=2013-03-24 #DATE2=`date -v "-1d" "+%Y-%m-%d"` # snap the source ssh root@tbh.lerctr.org zfs snapshot -r zroot@${DATE} # zfs copy the source to here. ssh root@tbh.lerctr.org 2>zfs-send.log "zfs send -v -R zroot@${DATE} " | \ mbuffer -m 16M 2>mbuffer.log | \ zfs recv -F -u -v -d zroot/backups/TBH3 2>zfs-recv.log # make sure we NEVER allow the backup stuff to automount. /sbin/zfs list -H -t filesystem -r zroot/backups/TBH3| \ awk '{printf "/sbin/zfs set canmount=noauto %s\n",$1}' | sh borg.lerctr.org /home/ler/bin $ borg.lerctr.org /home/ler/bin $ ssh tbh vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 384, 0, 202, 8, 202, 0, 0 UMA Zones: 1664, 0, 202, 0, 202, 0, 0 UMA Slabs: 80, 0, 363320, 44080, 2348572, 0, 0 UMA RCntSlabs: 88, 0, 4484, 16, 4484, 0, 0 UMA Hash: 256, 0, 7, 83, 82, 0, 0 4 Bucket: 32, 0, 1911, 36589, 5255345, 0, 0 6 Bucket: 48, 0, 9406, 3542, 878903, 0, 0 8 Bucket: 64, 0, 42, 3554, 298443, 11, 0 12 Bucket: 96, 0, 93, 2572, 166067, 0, 0 16 Bucket: 128, 0, 30447, 987, 301403, 0, 0 32 Bucket: 256, 0, 352, 4658, 1157489, 50, 0 64 Bucket: 512, 0, 13669, 995, 1113780,268080, 0 128 Bucket: 1024, 0, 3646, 822, 524977, 0, 0 256 Bucket: 2048, 0, 3648, 114, 482627,59834, 0 vmem btag: 56, 0, 208448, 49779, 758362,1821, 0 VM OBJECT: 256, 0, 98960, 1570, 4440323, 0, 0 RADIX NODE: 144, 0, 235166, 29650,22669417, 0, 0 MAP: 240, 0, 3, 61, 3, 0, 0 KMAP ENTRY: 128, 0, 10, 269, 10, 0, 0 MAP ENTRY: 128, 0, 11828, 24442,12463199, 0, 0 VMSPACE: 448, 0, 103, 311, 96786, 0, 0 fakepg: 104, 0, 0, 0, 0, 0, 0 mt_zone: 4112, 0, 368, 0, 368, 0, 0 16: 16, 0, 264961, 9382,56032463, 0, 0 32: 32, 0, 155626, 2874,55177821, 0, 0 64: 64, 0, 123597, 672111,53838666, 0, 0 128: 128, 0, 159107, 58792,82084329, 0, 0 256: 256, 0, 97004, 223276,48661927, 0, 0 512: 512, 0, 737, 3991,33323191, 0, 0 1024: 1024, 0, 1367, 1389, 2330023, 0, 0 2048: 2048, 0, 271, 449,16353342, 0, 0 4096: 4096, 0, 28044, 27448, 997566, 0, 0 SLEEPQUEUE: 80, 0, 613, 658, 613, 0, 0 64 pcpu: 8, 0, 1884, 1188, 1884, 0, 0 Files: 80, 0, 861, 1639, 5410116, 0, 0 rl_entry: 40, 0, 211, 1889, 211, 0, 0 TURNSTILE: 136, 0, 613, 307, 613, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0 MAC labels: 40, 0, 0, 0, 0, 0, 0 PROC: 1208, 0, 138, 114, 97778, 0, 0 THREAD: 1168, 0, 517, 95, 2429, 0, 0 cpuset: 72, 0, 270, 280, 426, 0, 0 cyclic_id_cache: 64, 0, 0, 0, 0, 0, 0 audit_record: 1248, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 6519810, 4080, 3257, 9746764, 0, 0 mbuf: 256, 6519810, 1025, 3278,259201769, 0, 0 mbuf_cluster: 2048, 1018718, 7337, 13, 7337, 0, 0 mbuf_jumbo_page: 4096, 509359, 0, 809,45525783, 0, 0 mbuf_jumbo_9k: 9216, 150921, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 84893, 0, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 2510, 116647, 0, 0 dtrace_state_cache: 4096, 0, 0, 0, 0, 0, 0 g_bio: 248, 0, 0, 1680,48127406, 0, 0 ttyinq: 160, 0, 120, 630, 2460, 0, 0 DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0 ttyoutq: 256, 0, 64, 671, 1279, 0, 0 ata_request: 336, 0, 0, 242, 42892, 0, 0 vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0 cryptop: 88, 0, 0, 0, 0, 0, 0 cryptodesc: 72, 0, 0, 0, 0, 0, 0 FPU_save_area: 512, 0, 0, 0, 0, 0, 0 taskq_zone: 48, 0, 0, 3071, 250215, 0, 0 taskq_zone: 48, 0, 0, 0, 0, 0, 0 VNODE: 472, 0, 113296, 62392, 2709794, 0, 0 VNODEPOLL: 112, 0, 355, 660, 357, 0, 0 BUF TRIE: 144, 0, 0, 105354, 0, 0, 0 S VFS Cache: 108, 0, 118644, 56881, 2428046, 0, 0 STS VFS Cache: 148, 0, 0, 0, 0, 0, 0 L VFS Cache: 328, 0, 1110, 144894, 804385, 0, 0 LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0 NAMEI: 1024, 0, 0, 172,30294048, 0, 0 DIRHASH: 1024, 0, 0, 0, 0, 0, 0 NCLNODE: 528, 0, 0, 0, 0, 0, 0 Mountpoints: 816, 0, 11, 19, 11, 0, 0 range_seg_cache: 64, 0, 59688, 6094, 4438296, 0, 0 zio_cache: 920, 0, 1, 42591,138793685, 0, 0 zio_link_cache: 48, 0, 0, 44073,97697152, 0, 0 zio_buf_512: 512, 0, 411934, 265850, 1785999, 0, 0 zio_data_buf_512: 512, 0, 154726, 20594, 797483, 0, 0 zio_buf_1024: 1024, 0, 3061, 175, 74166, 0, 0 zio_data_buf_1024: 1024, 0, 69831, 9029, 333451, 0, 0 zio_buf_1536: 1536, 0, 1372, 96, 93985, 0, 0 zio_data_buf_1536: 1536, 0, 29821, 3103, 190819, 0, 0 zio_buf_2048: 2048, 0, 523, 47, 281620, 0, 0 zio_data_buf_2048: 2048, 0, 17707, 2389, 128398, 0, 0 zio_buf_2560: 2560, 0, 390, 28, 22269, 0, 0 zio_data_buf_2560: 2560, 0, 12713, 28, 108546, 0, 0 zio_buf_3072: 3072, 0, 182, 22, 7255, 0, 0 zio_data_buf_3072: 3072, 0, 8798, 34, 79372, 0, 0 zio_buf_3584: 3584, 0, 206, 11, 50516, 0, 0 zio_data_buf_3584: 3584, 0, 6275, 11, 64145, 0, 0 zio_buf_4096: 4096, 0, 1552, 416,18817937, 0, 0 zio_data_buf_4096: 4096, 0, 4625, 6, 59910, 0, 0 zio_buf_5120: 5120, 0, 133, 9, 8048, 0, 0 zio_data_buf_5120: 5120, 0, 6326, 11, 77387, 0, 0 zio_buf_6144: 6144, 0, 86, 11, 4503, 0, 0 zio_data_buf_6144: 6144, 0, 4632, 13, 58064, 0, 0 zio_buf_7168: 7168, 0, 61, 13, 48012, 0, 0 zio_data_buf_7168: 7168, 0, 3090, 13, 44010, 0, 0 zio_buf_8192: 8192, 0, 24, 29, 1857848, 0, 0 zio_data_buf_8192: 8192, 0, 2418, 11, 42210, 0, 0 zio_buf_10240: 10240, 0, 49, 7, 3311, 0, 0 zio_data_buf_10240: 10240, 0, 3611, 15, 59310, 0, 0 zio_buf_12288: 12288, 0, 44, 14, 999494, 0, 0 zio_data_buf_12288: 12288, 0, 2340, 13, 53847, 0, 0 zio_buf_14336: 14336, 0, 23, 10, 4184, 0, 0 zio_data_buf_14336: 14336, 0, 1852, 15, 36058, 0, 0 zio_buf_16384: 16384, 0, 72595, 164, 3891311, 0, 0 zio_data_buf_16384: 16384, 0, 1707, 11, 33236, 0, 0 zio_buf_20480: 20480, 0, 201, 261, 832993, 0, 0 zio_data_buf_20480: 20480, 0, 1932, 12, 43873, 0, 0 zio_buf_24576: 24576, 0, 16, 11, 365481, 0, 0 zio_data_buf_24576: 24576, 0, 1852, 13, 31773, 0, 0 zio_buf_28672: 28672, 0, 9, 25, 731759, 0, 0 zio_data_buf_28672: 28672, 0, 1169, 10, 27692, 0, 0 zio_buf_32768: 32768, 0, 9, 14, 112680, 0, 0 zio_data_buf_32768: 32768, 0, 962, 13, 20775, 0, 0 zio_buf_36864: 36864, 0, 5, 14, 112387, 0, 0 zio_data_buf_36864: 36864, 0, 701, 13, 15139, 0, 0 zio_buf_40960: 40960, 0, 4, 11, 115602, 0, 0 zio_data_buf_40960: 40960, 0, 720, 10, 12360, 0, 0 zio_buf_45056: 45056, 0, 48, 12, 102919, 0, 0 zio_data_buf_45056: 45056, 0, 434, 14, 10533, 0, 0 zio_buf_49152: 49152, 0, 188, 12, 113099, 0, 0 zio_data_buf_49152: 49152, 0, 373, 10, 11797, 0, 0 zio_buf_53248: 53248, 0, 24, 13, 157215, 0, 0 zio_data_buf_53248: 53248, 0, 347, 15, 7265, 0, 0 zio_buf_57344: 57344, 0, 1, 11, 154125, 0, 0 zio_data_buf_57344: 57344, 0, 274, 14, 7110, 0, 0 zio_buf_61440: 61440, 0, 2, 10, 163599, 0, 0 zio_data_buf_61440: 61440, 0, 180, 11, 5565, 0, 0 zio_buf_65536: 65536, 0, 1, 11, 186705, 0, 0 zio_data_buf_65536: 65536, 0, 213, 11, 7273, 0, 0 zio_buf_69632: 69632, 0, 2, 14, 145850, 0, 0 zio_data_buf_69632: 69632, 0, 176, 8, 5232, 0, 0 zio_buf_73728: 73728, 0, 1, 13, 136031, 0, 0 zio_data_buf_73728: 73728, 0, 127, 12, 4816, 0, 0 zio_buf_77824: 77824, 0, 1, 22, 162862, 0, 0 zio_data_buf_77824: 77824, 0, 136, 14, 4207, 0, 0 zio_buf_81920: 81920, 0, 0, 149, 195810, 0, 0 zio_data_buf_81920: 81920, 0, 99, 15, 4936, 0, 0 zio_buf_86016: 86016, 0, 1, 216, 150971, 0, 0 zio_data_buf_86016: 86016, 0, 103, 14, 33989, 0, 0 zio_buf_90112: 90112, 0, 1, 120, 75823, 0, 0 zio_data_buf_90112: 90112, 0, 145, 11, 3517, 0, 0 zio_buf_94208: 94208, 0, 0, 20, 72296, 0, 0 zio_data_buf_94208: 94208, 0, 149, 11, 3315, 0, 0 zio_buf_98304: 98304, 0, 0, 19, 68257, 0, 0 zio_data_buf_98304: 98304, 0, 79, 11, 4092, 0, 0 zio_buf_102400: 102400, 0, 1, 19, 62463, 0, 0 zio_data_buf_102400: 102400, 0, 92, 12, 2940, 0, 0 zio_buf_106496: 106496, 0, 0, 15, 203444, 0, 0 zio_data_buf_106496: 106496, 0, 62, 14, 2869, 0, 0 zio_buf_110592: 110592, 0, 0, 10, 184072, 0, 0 zio_data_buf_110592: 110592, 0, 152, 12, 2463, 0, 0 zio_buf_114688: 114688, 0, 1, 12, 65538, 0, 0 zio_data_buf_114688: 114688, 0, 115, 15, 3609, 0, 0 zio_buf_118784: 118784, 0, 0, 13, 73524, 0, 0 zio_data_buf_118784: 118784, 0, 57, 12, 2092, 0, 0 zio_buf_122880: 122880, 0, 0, 23, 101816, 0, 0 zio_data_buf_122880: 122880, 0, 46, 15, 2131, 0, 0 zio_buf_126976: 126976, 0, 0, 13, 99774, 0, 0 zio_data_buf_126976: 126976, 0, 46, 15, 1927, 0, 0 zio_buf_131072: 131072, 0, 21, 97, 715685, 0, 0 zio_data_buf_131072: 131072, 0, 64623, 8, 1133068, 0, 0 lz4_ctx: 16384, 0, 0, 38, 2874956, 0, 0 sa_cache: 80, 0, 113220, 62680, 2719738, 0, 0 dnode_t: 744, 0, 369347, 391598, 903612, 0, 0 dmu_buf_impl_t: 224, 0, 459185, 519692, 3494852, 0, 0 arc_buf_hdr_t: 216, 0, 697777, 122573, 4738762, 0, 0 arc_buf_t: 72, 0, 541440, 121695, 5738782, 0, 0 zil_lwb_cache: 192, 0, 7, 5073, 90526, 0, 0 zfs_znode_cache: 368, 0, 113220, 62340, 2709584, 0, 0 pipe: 744, 0, 113, 177, 84427, 0, 0 procdesc: 128, 0, 0, 0, 0, 0, 0 ksiginfo: 112, 0, 1133, 2507,10649838, 0, 0 itimer: 352, 0, 1, 32, 1, 0, 0 KNOTE: 128, 0, 379, 1388,26313314, 0, 0 socket: 696, 523340, 243, 317, 827216, 0, 0 unpcb: 240, 523344, 152, 648, 171731, 0, 0 ipq: 56, 31879, 0, 2059, 385, 0, 0 udp_inpcb: 392, 523340, 19, 421, 567701, 0, 0 udpcb: 16, 523586, 19, 2993, 567701, 0, 0 tcp_inpcb: 392, 523340, 73, 457, 87773, 0, 0 tcpcb: 1024, 523340, 67, 245, 87773, 0, 0 tcptw: 88, 27810, 6, 1254, 29354, 0, 0 syncache: 160, 15375, 0, 800, 57377, 0, 0 hostcache: 136, 15370, 134, 620, 4951, 0, 0 tcpreass: 40, 63700, 0, 3100, 13421, 0, 0 sackhole: 32, 0, 0, 2125, 18903, 0, 0 sctp_ep: 1408, 523340, 0, 0, 0, 0, 0 sctp_asoc: 2352, 40000, 0, 0, 0, 0, 0 sctp_laddr: 48, 80012, 0, 1328, 8, 0, 0 sctp_raddr: 728, 80000, 0, 0, 0, 0, 0 sctp_chunk: 136, 400026, 0, 0, 0, 0, 0 sctp_readq: 104, 400026, 0, 0, 0, 0, 0 sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0 sctp_asconf: 40, 400000, 0, 0, 0, 0, 0 sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0 udplite_inpcb: 392, 523340, 0, 0, 0, 0, 0 ripcb: 392, 523340, 2, 58, 3, 0, 0 rtentry: 200, 0, 32, 408, 34, 0, 0 selfd: 56, 0, 356, 2200,137062686, 0, 0 SWAPMETA: 288, 2037438, 0, 0, 0, 0, 0 borg.lerctr.org /home/ler/bin $ last pid: 97813; load averages: 7.79, 8.41, 8.24 up 1+11:45:45 23:42:11 118 processes: 7 running, 109 sleeping, 2 stopped Mem: 982M Active, 1172M Inact, 13G Wired, 148M Cache, 637M Free ARC: 11G Total, 4394M MFU, 5813M MRU, 1029K Anon, 181M Header, 469M Other Swap: 18G Total, 18G Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 97782 boinc 1 155 i31 91992K 26008K CPU1 1 0:40 100.00% wcgrid_f 96977 boinc 1 155 i31 375M 165M STOP 4 11:30 61.87% wcgrid_c 89981 boinc 1 155 i31 99308K 63248K CPU5 5 91:25 51.76% wcgrid_f 92270 boinc 1 155 i31 98M 64556K CPU3 3 62:55 50.88% wcgrid_f 92269 boinc 1 155 i31 99372K 63324K CPU4 4 63:49 50.20% wcgrid_f 92271 boinc 1 155 i31 99340K 63288K CPU0 0 64:00 48.19% wcgrid_f 89772 boinc 1 155 i31 99888K 63832K CPU7 7 92:52 41.36% wcgrid_f 89781 boinc 1 155 i31 99732K 63588K nanslp 5 94:54 38.18% wcgrid_f 85619 root 1 52 0 181M 97232K select 3 2:56 21.19% perl5 97806 root 1 28 0 44600K 8028K kqread 0 0:00 0.78% indexer- 837 clamav 3 20 0 486M 399M uwait 5 12:42 0.20% clamd 70563 boinc 2 155 i31 101M 51436K nanslp 7 202:02 0.00% setiatho 68437 boinc 2 155 i31 105M 52516K nanslp 3 185:45 0.00% setiatho 83114 boinc 2 155 i31 105M 51880K nanslp 4 103:15 0.00% setiatho 85300 boinc 2 155 i31 101M 51460K nanslp 4 78:55 0.00% setiatho 86148 boinc 2 155 i31 105M 51492K nanslp 3 71:54 0.00% setiatho 87150 boinc 2 155 i31 95112K 46940K nanslp 4 30:37 0.00% setiatho 87451 boinc 2 155 i31 95112K 46224K nanslp 3 28:07 0.00% setiatho borg.lerctr.org /home/ler/bin $ Where to now? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688