From owner-svn-src-user@freebsd.org Tue Dec 15 07:31:01 2020 Return-Path: Delivered-To: svn-src-user@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 865264AB835 for ; Tue, 15 Dec 2020 07:31:01 +0000 (UTC) (envelope-from pho@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cw90j3F9hz3Fwj; Tue, 15 Dec 2020 07:31:01 +0000 (UTC) (envelope-from pho@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6245810683; Tue, 15 Dec 2020 07:31:01 +0000 (UTC) (envelope-from pho@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0BF7V178012263; Tue, 15 Dec 2020 07:31:01 GMT (envelope-from pho@FreeBSD.org) Received: (from pho@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0BF7V0sD012261; Tue, 15 Dec 2020 07:31:00 GMT (envelope-from pho@FreeBSD.org) Message-Id: <202012150731.0BF7V0sD012261@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: pho set sender to pho@FreeBSD.org using -f From: Peter Holm Date: Tue, 15 Dec 2020 07:31:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org Subject: svn commit: r368655 - user/pho/stress2/misc X-SVN-Group: user X-SVN-Commit-Author: pho X-SVN-Commit-Paths: user/pho/stress2/misc X-SVN-Commit-Revision: 368655 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Dec 2020 07:31:01 -0000 Author: pho Date: Tue Dec 15 07:31:00 2020 New Revision: 368655 URL: https://svnweb.freebsd.org/changeset/base/368655 Log: Added more swapoff test scenarios. Added: user/pho/stress2/misc/swapoff2.sh (contents, props changed) user/pho/stress2/misc/swapoff4.sh (contents, props changed) user/pho/stress2/misc/swapoff5.sh (contents, props changed) Added: user/pho/stress2/misc/swapoff2.sh ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/pho/stress2/misc/swapoff2.sh Tue Dec 15 07:31:00 2020 (r368655) @@ -0,0 +1,79 @@ +#!/bin/sh + +# +# Copyright (c) 2017 Dell EMC Isilon +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. +# +# $FreeBSD$ +# + +[ `id -u ` -ne 0 ] && echo "Must be root!" && exit 1 + +# swapoff(8) with swap file gone missing (known issue). + +# panic: swap_pager_force_pagein: read from swap failed +# https://people.freebsd.org/~pho/stress/log/swapoff2.txt + +. ../default.cfg + +set -e +mount | grep $mntpoint | grep -q /dev/md && umount -f $mntpoint +mdconfig -l | grep -q md$mdstart && mdconfig -d -u $mdstart +mdconfig -a -t swap -s 2g -u $mdstart +bsdlabel -w md$mdstart auto +newfs $newfs_flags md${mdstart}$part > /dev/null +mount /dev/md${mdstart}$part $mntpoint +set +e + +: ${size:=2m} +dd if=/dev/zero of=$mntpoint/swapfile bs=$size count=1 2>&1 | \ + egrep -v "records|transferred" +ls -lh $mntpoint/swapfile +swapoff -a +md2=$((mdstart + 1)) +mdconfig -l | grep -q md$md2 && mdconfig -d -u $md2 +mdconfig -a -t vnode -f $mntpoint/swapfile -u $md2 +set +e +swapon /dev/md$md2 + +su $testuser -c 'cd ../testcases/swap; nice ./swap -t 2m -i 20 -h -l 100' & +for i in `jot 20`; do + [ `swapinfo | tail -1 | awk '{print $3}'` -gt 0 ] && break + sleep 5 +done +umount -f $mntpoint +swapoff /dev/md$md2 + +while pgrep -q swap; do pkill -9 swap; done +wait + +n=0 +while mount | grep $mntpoint | grep -q /dev/md; do + umount $mntpoint || sleep 1 + n=$((n + 1)) + [ $n -gt 300 ] && { echo FAIL; exit 1; } +done +mdconfig -d -u $mdstart +swapon -a +exit 0 Added: user/pho/stress2/misc/swapoff4.sh ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/pho/stress2/misc/swapoff4.sh Tue Dec 15 07:31:00 2020 (r368655) @@ -0,0 +1,85 @@ +#!/bin/sh + +# +# SPDX-License-Identifier: BSD-2-Clause-FreeBSD +# +# Copyright (c) 2020 Peter Holm +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. +# +# $FreeBSD$ +# + +# A variation of Mark's swapoff3.sh + +# "panic: Last close with active requests" seen: +# https://people.freebsd.org/~pho/stress/log/mark134.txt +# Fixed by r358024-6 + +. ../default.cfg +[ `id -u` -ne 0 ] && echo "Must be root!" && exit 1 +dir=`dirname $diskimage` +[ `df -k $dir | tail -1 | awk '{print $4}'` -lt \ + $((20 * 1024 * 1024)) ] && + { echo "Need 20GB on $dir"; exit 0; } + + +set -e +md1=$mdstart +md2=$((md1 + 1)) +[ `sysctl -n vm.swap_total` -gt 0 ] && { swapoff -a > /dev/null; off=1; } +truncate -s 10G $dir/swap1 +mdconfig -a -t vnode -f $dir/swap1 -u $md1 +swapon /dev/md$md1 +truncate -s 10G $dir/swap2 +mdconfig -a -t vnode -f $dir/swap2 -u $md2 +swapon /dev/md$md2 +set +e + +export swapINCARNATIONS=25 +(cd ../testcases/swap; ./swap -t 4m -i 20 -l 100) & +for i in `jot 90`; do + used=`swapinfo | tail -1 | awk '{print $3}'` + [ $used -gt 0 ] && break + sleep 2 +done + +n=0 +while pgrep -q swap; do + for dev in /dev/md$md1 /dev/md$md2; do + for i in `jot 3`; do + swapoff $dev && break + done + swapon $dev + n=$((n + 1)) + done +done +[ $n -lt 100 ] && { echo "Only performed $n swapoffs"; s=1; } || s=0 +wait +for dev in /dev/md$md1 /dev/md$md2; do + swapoff $dev +done +mdconfig -d -u $md1 +mdconfig -d -u $md2 +rm -f $dir/swap1 $dir/swap2 +[ $off ] && swapon -a > /dev/null +exit 0 Added: user/pho/stress2/misc/swapoff5.sh ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/pho/stress2/misc/swapoff5.sh Tue Dec 15 07:31:00 2020 (r368655) @@ -0,0 +1,182 @@ +#!/bin/sh + +# +# SPDX-License-Identifier: BSD-2-Clause-FreeBSD +# +# Copyright (c) 2020 Mark Johnston +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. +# +# $FreeBSD$ +# + +# Variation of swapoff3.sh + +# Deadlock seen: +# https://people.freebsd.org/~pho/stress/log/log0005.txt + +. ../default.cfg +[ `id -u` -ne 0 ] && echo "Must be root!" && exit 1 +dir=`dirname $diskimage` +[ `df -k $dir | tail -1 | awk '{print $4}'` -lt \ + $((20 * 1024 * 1024)) ] && + { echo "Need 20GB on $dir"; exit 0; } + +cat > /tmp/swapoff5.c < +#include +#include + +#include +#include +#include +#include + +int +main(void) +{ + char *addr, *naddr, *res; + size_t i, len, swap, vsz; + u_int free; + + vsz = sizeof(free); + if (sysctlbyname("vm.stats.vm.v_free_count", &free, &vsz, NULL, 0) != 0) + err(1, "sysctl(vm.stats.vm.v_free_count)"); + vsz = sizeof(swap); + if (sysctlbyname("vm.swap_total", &swap, &vsz, NULL, 0) != 0) + err(1, "sysctl(vm.swap_total)"); + + len = (size_t)free * PAGE_SIZE + swap / 4; + addr = mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_ANON | MAP_PRIVATE, + -1, 0); + if (addr == MAP_FAILED) + err(1, "mmap"); + + res = malloc(howmany(len, PAGE_SIZE)); + if (res == NULL) + err(1, "malloc"); + + for (i = 0; i < len; i += PAGE_SIZE) + addr[i] = 1; + + for (;;) { + if (mincore(addr, len, res) != 0) + err(1, "mincore"); + for (i = 0; i < howmany(len, PAGE_SIZE); i++) + if ((res[i] & MINCORE_INCORE) == 0) { + naddr = addr + i * PAGE_SIZE; + if (munmap(naddr, PAGE_SIZE) != 0) + err(1, "munmap"); + if (mmap(naddr, PAGE_SIZE, + PROT_READ | PROT_WRITE, + MAP_ANON | MAP_PRIVATE | MAP_FIXED, + -1, 0) == MAP_FAILED) + err(1, "mmap"); + assert(*naddr == 0); + *naddr = 1; + } + } + + return (0); +} +EOF +mycc -o /tmp/swapoff5 -Wall -Wextra -O2 -g /tmp/swapoff5.c || exit + +cat > /tmp/swapoff5x.c < + +#include +#include +#include +#include + +int +main(int argc, char *argv[]) +{ + time_t start; + int n, r; + char off1[80], off2[80], on1[80], on2[80]; + + if ( argc != 3) { + fprintf(stderr, "Usage: %s \n", argv[0]); + exit(1); + } + + snprintf(off1, sizeof(off1), "swapoff %s", argv[1]); + snprintf(off2, sizeof(off2), "swapoff %s", argv[2]); + snprintf(on1, sizeof(on1), "swapon %s", argv[1]); + snprintf(on2, sizeof(on2), "swapon %s", argv[2]); + + start = time(NULL); + if (mlockall(MCL_CURRENT | MCL_FUTURE) == -1) + err(1, "mlockall(MCL_CURRENT) | MCL_FUTURE"); + n = 0; + while (time(NULL) - start < 240) { + if ((r = system(off1)) != 0) + errx(1, "%s failed with %d", off1, r); + if ((r = system(on1)) != 0) + errx(1, "%s failed with %d", on1, r); + if ((r = system(off2)) != 0) + errx(1, "%s failed with %d", off2, r); + if ((r = system(on2)) != 0) + errx(1, "%s failed with %d", on2, r); + n++; + } + if (n < 100) + fprintf(stderr, "%d swapoffs\n", n); + + return (0); +} +EOF +mycc -o /tmp/swapoff5x -Wall -Wextra -O2 -g /tmp/swapoff5x.c || exit + +set -e +md1=$mdstart +md2=$((md1 + 1)) +[ `sysctl -n vm.swap_total` -gt 0 ] && { swapoff -a > /dev/null; off=1; } +truncate -s 10G $dir/swap1 +mdconfig -a -t vnode -f $dir/swap1 -u $md1 +swapon /dev/md$md1 +truncate -s 10G $dir/swap2 +mdconfig -a -t vnode -f $dir/swap2 -u $md2 +swapon /dev/md$md2 +set +e + +timeout 4m /tmp/swapoff5 & +for i in `jot 60`; do + n=`swapinfo | tail -1 | awk '{print $3}'` + [ $n -gt 0 ] && break + sleep 1 +done + +/tmp/swapoff5x /dev/md$md1 /dev/md$md2 +kill $! > /dev/null 2>&1 +wait +for dev in /dev/md$md1 /dev/md$md2; do + swapoff $dev +done +mdconfig -d -u $md1 +mdconfig -d -u $md2 +rm -f $dir/swap1 $dir/swap2 /tmp/swapoff5 /tmp/swapoff5.c /tmp/swapoff5x \ + /tmp/swapoff5x.c +[ $off ] && swapon -a > /dev/null +exit $s From owner-svn-src-user@freebsd.org Thu Dec 17 05:24:16 2020 Return-Path: Delivered-To: svn-src-user@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 21A484AB219 for ; Thu, 17 Dec 2020 05:24:16 +0000 (UTC) (envelope-from bounces+17430347-64d8-svn-src-user=freebsd.org@em4842.fdc-k.africa) Received: from xtrwsqzn.outbound-mail.sendgrid.net (xtrwsqzn.outbound-mail.sendgrid.net [167.89.100.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CxL5W4rkyz4v2B for ; Thu, 17 Dec 2020 05:24:15 +0000 (UTC) (envelope-from bounces+17430347-64d8-svn-src-user=freebsd.org@em4842.fdc-k.africa) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fdc-k.africa; h=content-transfer-encoding:content-type:from:mime-version:to:subject:list-unsubscribe; s=s1; bh=RtwxoVhSKwbiyW087NaVmzeZC1L7yMHfbNeSt9j5Inw=; b=CsSFkuq s+NZcSBJ/qzJUrkvj7GePUoDrBGlfy+Djw8smmByvIjigmEwR7ZDS0AWTD2S8Uau uIHuaVnlQwIaqxF0/ZuWeOQSV/A+qUTRTfTL0fsg3NIAR2hnEuPVEHXQFtLb15b4 NXWQXY1xu3lTRNeEsOWQWquFYyUu3uvt8MRA= Received: by filter2142p1las1.sendgrid.net with SMTP id filter2142p1las1-9889-5FDADFE6-1 2020-12-17 04:34:47.005234573 +0000 UTC m=+3168.656928624 Received: from MTc0MzAzNDc (unknown) by ismtpd0003p1sjc2.sendgrid.net (SG) with HTTP id OBygwgnbQg-qhyO6V0qP_w Thu, 17 Dec 2020 04:34:45.848 +0000 (UTC) Date: Thu, 17 Dec 2020 04:35:05 +0000 (UTC) From: "FDC Training" To: svn-src-user@freebsd.org Message-ID: Subject: Invitation to GIS for Monitoring and Evaluation training JAN 2021 X-SG-EID: 0ni5TANSIXULvtN0oVthSHlGelr5laoPfbUaGoL9TsBqyTGNKLocEyBTfwHnkFvovUZrowQna/q252 JK7MMsI8NsXkNKXlXhMd/mVLb+HPKTWe95dc/bTn/5oktBB2SnkZRyobBx3aQD219QjdMZ1gm1Gq8B o4sz97MRB7BFX2XhOgEvS9JbPBO6y/eGcDJ35yWYNmKJj6QzphhgMCP2ZQOumwgoKeRgyq0ZlWHJE1 A= X-Entity-ID: lgZohGoWIjvYsKzlUpYsxA== X-Rspamd-Queue-Id: 4CxL5W4rkyz4v2B X-Spamd-Bar: ++++++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=fdc-k.africa header.s=s1 header.b=CsSFkuq ; dmarc=pass (policy=none) header.from=fdc-k.africa; spf=pass (mx1.freebsd.org: domain of bounces@em4842.fdc-k.africa designates 167.89.100.226 as permitted sender) smtp.mailfrom=bounces@em4842.fdc-k.africa X-Spamd-Result: default: False [6.05 / 15.00]; R_SPF_ALLOW(0.00)[+ip4:167.89.0.0/17:c]; MV_CASE(0.50)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[fdc-k.africa:+]; DMARC_POLICY_ALLOW(0.00)[fdc-k.africa,none]; FORGED_SENDER(0.30)[training@fdc-k.africa,bounces@em4842.fdc-k.africa]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[167.89.100.226:from]; ASN(0.00)[asn:11377, ipnet:167.89.96.0/20, country:US]; MIME_TRACE(0.00)[0:~]; TAGGED_FROM(0.00)[17430347-64d8-svn-src-user=freebsd.org]; ARC_NA(0.00)[]; R_DKIM_ALLOW(0.00)[fdc-k.africa:s=s1]; FROM_NEQ_ENVFROM(0.00)[training@fdc-k.africa,bounces@em4842.fdc-k.africa]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.97)[0.971]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[167.89.100.226:from:127.0.2.255]; NEURAL_SPAM_MEDIUM(0.99)[0.990]; BAD_REP_POLICIES(0.10)[]; NEURAL_SPAM_LONG(1.00)[1.000]; MIME_HTML_ONLY(0.20)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[167.89.100.226:from]; RBL_SENDERSCORE(2.00)[167.89.100.226:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[svn-src-user] X-Spam: Yes MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Dec 2020 05:24:16 -0000