From owner-freebsd-stable@freebsd.org Tue Sep 22 16:33:28 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 61D003EEE55 for ; Tue, 22 Sep 2020 16:33:28 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bwn1M3Vx3z4H6m for ; Tue, 22 Sep 2020 16:33:27 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-io1-xd2d.google.com with SMTP id z25so20272686iol.10 for ; Tue, 22 Sep 2020 09:33:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=ET+Mx8FmmdmDRF2W0O+fOzHf1RvLGchqFMeqNu8yUzw=; b=iSYickbB4Sx2vDZ40C4muL3JAnrRFb1uNkSkjdiZoWseWYrOZiyfrs/d1D+HFJkid0 UFcX77C7k4pXKbcR9XzFuZ9ai/yoWUkU29fmLK1mLKYOQc6iQBa2PEeLVu5ZAs2ncgC1 kToZSqC7IJbiFDtGrRoaQ0H/hqOTgCJMNRMYIO4YYbijKa6E1Pf7gXPG7oPiHOfPB6In HlZuRcrotAu6MnDLmWxChUipcdxXOOTVOpRwBxB14Q364+dmZ+mUovtt+EKM4U6nhGPv 0mbvj6oakLGBMUmn3s3EkzModm0UNvFzDM6zmRzBbvu5r3AJryeh3rWfMgKbj829t9f6 9QVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=ET+Mx8FmmdmDRF2W0O+fOzHf1RvLGchqFMeqNu8yUzw=; b=jbj8t/CgumsOkR4AYXAwg9C5wWBV0n+GS1x013jPdZ4Ex22WEKgpbQjXVmuOj9XAgu hmxMMuH8nGXoIFeRPRWqWf/U7AdiW9YihO+4tNkqujwL5GPWKNJ/Wcd+vr5iR+cWajPz Bxx0Ifty20lLJz9kejDwvwg58Km4d45JMI29nLNTcEVi2+r8YK6rj7mqQpx/Qboi5Gmm /alUJi2FHmBJWX+TTUUJfLuHSwkjmBLjnqV00MHX5gb0SloJWD5uOAf7+7CGScImOBcF AB+uyILZsJxfymDztbqltQMZIYR65KW9XwbeTTWmGZDObPyWLSaToywk98E1829YZPh3 yx/g== X-Gm-Message-State: AOAM530dMjCXXlF5y3JH4OTesT8IA9vqWGS78BuwtVMSik9zPkN61Yai ezcj7/+BAFDkbljfYgeA2DdNe/CRrATJtg== X-Google-Smtp-Source: ABdhPJwY0ySEV0n/eCGkWoDqiK+0giCXeuPSKHTVRtZNDVrfkziqYloNGNyUlzpfSYn/146HdlgfIA== X-Received: by 2002:a05:6638:224e:: with SMTP id m14mr4795453jas.101.1600792406098; Tue, 22 Sep 2020 09:33:26 -0700 (PDT) Received: from raichu (toroon0560w-lp130-01-174-88-77-103.dsl.bell.ca. [174.88.77.103]) by smtp.gmail.com with ESMTPSA id h2sm7607967ioj.5.2020.09.22.09.33.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2020 09:33:25 -0700 (PDT) Sender: Mark Johnston Date: Tue, 22 Sep 2020 12:33:19 -0400 From: Mark Johnston To: Peter Cc: freebsd-stable@freebsd.org Subject: Re: How to free used Swap-Space? Message-ID: <20200922163319.GA70673@raichu> References: <20200922160801.GA19535@gate.oper.dinoex.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200922160801.GA19535@gate.oper.dinoex.org> X-Rspamd-Queue-Id: 4Bwn1M3Vx3z4H6m X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=iSYickbB; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::d2d as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-1.20 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.51)[-0.509]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.993]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.001]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::d2d:from]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2020 16:33:28 -0000 On Tue, Sep 22, 2020 at 06:08:01PM +0200, Peter wrote: > Hi all, > > my machine should use about 3-4, maybe 5 GB swapspace. Today I found > it suddenly uses 8 GB (which is worryingly near the configured 10G). > > I stopped all the big suckers - nothing found. > I stopped all the jails - no success. > I brought it down to singleuser: it tried to swapoff, but failed. > > I unmounted all filesystems, exported all pools, detached all geli, > and removed most of the netgraphs. Swap is still occupied. > > Machine is now running only the init and a shell processes, has > almost no filesystems mounted, has mostly native networks only, and > this still occupies 3 GB of swap which cannot be released. > > What is going on, what is doing this, and how can I get this swapspace > released?? Do you have any shared memory segments lingering? ipcs -a will show SysV shared memory usage. For POSIX shared memory, in 11.4 we do not have any good way of listing objects, but "vmstat -m | grep shmfd" will at least show whether any are allocated. If those don't turn anything up then it's possible that there's a swap leak. Do you use any DRM graphics drivers on this system? > > It is 11.4-RELEASE-p3 amd64. > > > Script started on Mon Sep 21 05:43:20 2020 > root@edge# ps axlww > UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND > 0 0 0 0 -16 0 0 752 swapin DLs - 291:32.41 [kernel] > 0 1 0 0 20 0 5416 248 wait ILs - 0:00.22 /sbin/init -- > 0 2 0 0 -16 0 0 16 ftcl DL - 0:00.00 [ftcleanup] > 0 3 0 0 -16 0 0 16 crypto_w DL - 0:00.00 [crypto] > 0 4 0 0 -16 0 0 16 crypto_r DL - 0:00.00 [crypto returns] > 0 5 0 0 -16 0 0 32 - DL - 11:41.94 [cam] > 0 6 0 0 -8 0 0 80 t->zthr_ DL - 13:07.13 [zfskern] > 0 7 0 0 -16 0 0 16 waiting_ DL - 0:00.00 [sctp_iterator] > 0 8 0 0 -16 0 0 16 - DL - 2:05.20 [rand_harvestq] > 0 9 0 0 -16 0 0 16 - DL - 0:00.04 [soaiod1] > 0 10 0 0 155 0 0 64 - RNL - 17115:06.48 [idle] > 0 11 0 0 -52 0 0 352 - WL - 49:05.30 [intr] > 0 12 0 0 -16 0 0 64 sleep DL - 16:28.51 [ng_queue] > 0 13 0 0 -8 0 0 48 - DL - 23:10.60 [geom] > 0 14 0 0 -16 0 0 16 seqstate DL - 0:00.00 [sequencer 00] > 0 15 0 0 -68 0 0 160 - DL - 0:23.64 [usb] > 0 16 0 0 -16 0 0 16 - DL - 0:00.04 [soaiod2] > 0 17 0 0 -16 0 0 16 - DL - 0:00.04 [soaiod3] > 0 18 0 0 -16 0 0 16 - DL - 0:00.04 [soaiod4] > 0 19 0 0 -16 0 0 16 idle DL - 0:00.83 [enc_daemon0] > 0 20 0 0 -16 0 0 48 psleep DL - 12:07.72 [pagedaemon] > 0 21 0 0 20 0 0 16 psleep DL - 4:12.41 [vmdaemon] > 0 22 0 0 155 0 0 16 pgzero DNL - 0:00.00 [pagezero] > 0 23 0 0 -16 0 0 64 psleep DL - 0:23.50 [bufdaemon] > 0 24 0 0 20 0 0 16 - DL - 0:04.21 [bufspacedaemon] > 0 25 0 0 16 0 0 16 syncer DL - 0:32.48 [syncer] > 0 26 0 0 -16 0 0 16 vlruwt DL - 0:02.31 [vnlru] > 0 27 0 0 -16 0 0 16 - DL - 7:11.58 [racctd] > 0 157 0 0 20 0 0 16 geli:w DL - 0:22.03 [g_eli[0] ada1p2] > 0 158 0 0 20 0 0 16 geli:w DL - 0:22.77 [g_eli[1] ada1p2] > 0 159 0 0 20 0 0 16 geli:w DL - 0:31.08 [g_eli[2] ada1p2] > 0 160 0 0 20 0 0 16 geli:w DL - 0:29.41 [g_eli[3] ada1p2] > 0 70865 1 0 20 0 7076 3104 wait Ss v0 0:00.21 -sh (sh) > 0 71135 70865 0 20 0 6392 2308 select S+ v0 0:00.00 script > 0 71136 71135 0 23 0 7076 3068 wait Ss 0 0:00.00 /bin/sh -i > 0 71142 71136 0 23 0 6928 2584 - R+ 0 0:00.00 ps axlww > > root@edge# df > Filesystem 512-blocks Used Avail Capacity Mounted on > /dev/ada3p3 1936568 860864 920784 48% / > devfs 2 2 0 100% /dev > procfs 8 8 0 100% /proc > /dev/ada3p4 3099192 1184896 1666368 42% /usr > /dev/ada3p5 580344 8112 525808 2% /var > > root@edge# pstat -s > Device 512-blocks Used Avail Capacity > /dev/ada1p2.eli 10485760 5839232 4646528 56% > > root@edge# top | cat > last pid: 71147; load averages: 0.19, 0.08, 0.09 up 3+03:21:00 05:44:12 > 5 processes: 1 running, 4 sleeping > > Mem: 9732K Active, 10M Inact, 882M Laundry, 1920M Wired, 10M Buf, 1023M Free > ARC: 335K Total, 16K MFU, 304K MRU, 15K Header > 320K Compressed, 2944K Uncompressed, 9.20:1 Ratio > Swap: 5120M Total, 2851M Used, 2269M Free, 55% Inuse > > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > 70865 root 1 20 0 7076K 3104K wait 2 0:00 0.00% sh > 71135 root 1 20 0 6392K 2308K select 1 0:00 0.00% script > 71136 root 1 20 0 7076K 3068K wait 2 0:00 0.00% sh > 71146 root 1 20 0 7928K 2980K CPU0 0 0:00 0.00% top > 71147 root 1 20 0 6300K 2088K piperd 1 0:00 0.00% cat > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"