From owner-svn-src-user@FreeBSD.ORG Mon Jun 1 05:43:28 2015 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CB8E7907; Mon, 1 Jun 2015 05:43:28 +0000 (UTC) (envelope-from pho@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ACA49127B; Mon, 1 Jun 2015 05:43:28 +0000 (UTC) (envelope-from pho@FreeBSD.org) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id t515hSBt047228; Mon, 1 Jun 2015 05:43:28 GMT (envelope-from pho@FreeBSD.org) Received: (from pho@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id t515hSWx047226; Mon, 1 Jun 2015 05:43:28 GMT (envelope-from pho@FreeBSD.org) Message-Id: <201506010543.t515hSWx047226@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: pho set sender to pho@FreeBSD.org using -f From: Peter Holm Date: Mon, 1 Jun 2015 05:43:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org Subject: svn commit: r283865 - user/pho/stress2/tools X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2015 05:43:29 -0000 Author: pho Date: Mon Jun 1 05:43:27 2015 New Revision: 283865 URL: https://svnweb.freebsd.org/changeset/base/283865 Log: Added tool for memory leak detection and tool for splitting tests across different hosts. Sponsored by: EMC / Isilon storage division Added: user/pho/stress2/tools/splitall.sh (contents, props changed) user/pho/stress2/tools/vmstat.sh (contents, props changed) Added: user/pho/stress2/tools/splitall.sh ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/pho/stress2/tools/splitall.sh Mon Jun 1 05:43:27 2015 (r283865) @@ -0,0 +1,60 @@ +#/bin/sh + +# +# Copyright (c) 2015 EMC Corp. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. +# +# $FreeBSD$ +# + +# A way to split all.sh across different hosts. +# For example "./splitall.sh 2 3" will run the second third of the tests. + +# Split the test list up in n parts and test one of them. + +[ $# -ne 2 ] && echo "Usage $0 " && exit 1 + +pno=$1 +parts=$2 + +cd ../misc +exclude=`sed -n '/^# Start of list/,/^# End of list/p' < all.sh | + cat - all.exclude 2>/dev/null | + grep "\.sh" | awk '{print $2}'` + +list=$(echo `ls *.sh` | sed "s/all\.sh//; s/cleanup\.sh//") + +lst="" +for i in $list; do + echo $exclude | grep -qw $i && continue + lst="$lst $i" +done +n=`echo $lst | wc -w` +(cd /tmp; echo $lst | tr ' ' '\n' | split -d -l $((n / parts)) - str) +part=`printf "/tmp/str%02d" $((pno - 1))` +plist=`cat $part | tr '\n' ' '` +rm -f /tmp/str0? +echo "./all.sh -on $plist" +sleep 10 +./all.sh -on $plist Added: user/pho/stress2/tools/vmstat.sh ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/pho/stress2/tools/vmstat.sh Mon Jun 1 05:43:27 2015 (r283865) @@ -0,0 +1,77 @@ +#!/bin/sh + +# +# Copyright (c) 2015 EMC Corp. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. +# +# $FreeBSD$ +# + +# Memory leak detector: run vmstat -m & -z in a loop + +[ $# -eq 1 ] && debug="-v debug=1" +export LANG=en_US.ISO8859-1 +OIFS=$IFS +while true; do + # Type InUse MemUse + vmstat -m | sed 1d | \ + sed 's/\(.* \)\([0-9][0-9]*\) *\(.*\)K .*/\1:\2:\3/' | \ + while IFS=: read -r p1 p2 p3; do + name=`echo $p1 | sed 's/^ *//;s/ *$//'` + memuse=$((p3 * 1024)) + [ "$memuse" -ne 0 ] && echo "vmstat -m $name,$memuse" + done + + # ITEM SIZE LIMIT USED + IFS=OIFS + vmstat -z | sed "1,2d;/^$/d" | while read l; do + IFS=':,' + set $l + size=$2 + used=$4 + [ "$used" -ne 0 ] && + echo "vmstat -z $1,$((size * used))" + done + sleep 10 +done | awk $debug -F, ' +{ +# Pairs of "name, value" are passed to this awk script + name=$1; + size=$2; + if (size > s[name]) { + if (++n[name] > 60) { + cmd="date '+%T'"; + cmd | getline t; + close(cmd); + printf "%s \"%s\" %'\''dK\r\n", t, + name, size / 1024; + n[name] = 0; + } + s[name] = size; + if (debug == 1 && n[name] > 1) + printf "%s, size %d, count %d\r\n", + name, s[name], n[name] + } else if (size < s[name] && n[name] > 0) + n[name]-- +}'