From owner-freebsd-current@freebsd.org Tue Mar 3 13:21:52 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2386A26F98B for ; Tue, 3 Mar 2020 13:21:52 +0000 (UTC) (envelope-from trond.endrestol@ximalas.info) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 48WyMy68dPz4Pnd for ; Tue, 3 Mar 2020 13:21:50 +0000 (UTC) (envelope-from trond.endrestol@ximalas.info) Received: by mailman.nyi.freebsd.org (Postfix) id 93EB526F98A; Tue, 3 Mar 2020 13:21:50 +0000 (UTC) Delivered-To: current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 92B2A26F989 for ; Tue, 3 Mar 2020 13:21:50 +0000 (UTC) (envelope-from trond.endrestol@ximalas.info) Received: from enterprise.ximalas.info (enterprise.ximalas.info [IPv6:2001:700:1100:1::8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "ximalas.info", Issuer "Hostmaster ximalas.info" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 48WyMw4KCTz4Phf for ; Tue, 3 Mar 2020 13:21:48 +0000 (UTC) (envelope-from trond.endrestol@ximalas.info) Received: from enterprise.ximalas.info (Ximalas@localhost [127.0.0.1]) by enterprise.ximalas.info (8.15.2/8.15.2) with ESMTPS id 023DLbw0011829 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 3 Mar 2020 14:21:37 +0100 (CET) (envelope-from trond.endrestol@ximalas.info) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ximalas.info; s=default; t=1583241697; bh=PGXZlu9ausJ0zMJ1/Z08TIsoT7SUxYU7g+zh4o0+CqY=; h=Date:From:To:cc:Subject:In-Reply-To:References; b=JviiU+bLjfRVApqn6HqafyCkHb++CM0drrW+pkClXHheiZmZOCJ1KIMoTy/HNsm87 TJxGdECxn5NTyVhBcrxvXmxyfc5BkAmRWDpO2nOjGl0EvBqfAmzId+EOlDCtQxiHPZ UGTMs26iH5Fdz6GiT7GRTqZBaVr4KAHCi8MBouk2K9v92Ei6i4IdVoot9CYNXkbSd7 feUZsd24uLNmz2499Gf7RH1BSqAgs2yiYEWRaGvoQx/wn6ilZoKgBzTHMSc3o6SRz8 uZZ2BOaBoEbiCV3p4B7FWo6HaqPo1Cz+06hHJ62H74E6jR35IaXhm6QKEhBXo955XZ sLZ2fwc82pHBg== Received: from localhost (trond@localhost) by enterprise.ximalas.info (8.15.2/8.15.2/Submit) with ESMTP id 023DLbOj011826; Tue, 3 Mar 2020 14:21:37 +0100 (CET) (envelope-from trond.endrestol@ximalas.info) X-Authentication-Warning: enterprise.ximalas.info: trond owned process doing -bs Date: Tue, 3 Mar 2020 14:21:37 +0100 (CET) From: =?UTF-8?Q?Trond_Endrest=C3=B8l?= Sender: Trond.Endrestol@ximalas.info To: Yuri Pankov cc: current@freebsd.org Subject: Re: can't build rust -- out of swap space In-Reply-To: <993c159b-b9e0-cd47-5df6-73288748ae13@fastmail.com> Message-ID: References: <993c159b-b9e0-cd47-5df6-73288748ae13@fastmail.com> User-Agent: Alpine 2.22 (BSF 395 2020-01-19) OpenPGP: url=http://ximalas.info/about/tronds-openpgp-public-key MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,URIBL_RED autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on enterprise.ximalas.info X-Rspamd-Queue-Id: 48WyMw4KCTz4Phf X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ximalas.info header.s=default header.b=JviiU+bL; dmarc=pass (policy=none) header.from=ximalas.info; spf=pass (mx1.freebsd.org: domain of trond.endrestol@ximalas.info designates 2001:700:1100:1::8 as permitted sender) smtp.mailfrom=trond.endrestol@ximalas.info X-Spamd-Result: default: False [0.69 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.95)[-0.947,0]; R_DKIM_ALLOW(0.00)[ximalas.info:s=default]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(0.00)[+a:c]; NEURAL_HAM_LONG(-0.91)[-0.906,0]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; HAS_XAW(0.00)[]; URIBL_RED(3.50)[ximalas.info.multi.uribl.com]; BAD_REP_POLICIES(0.10)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ximalas.info:+]; CTYPE_MIXED_BOGUS(1.00)[]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(0.00)[ximalas.info,none]; FREEMAIL_TO(0.00)[fastmail.com]; HAS_ANON_DOMAIN(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:224, ipnet:2001:700::/32, country:NO]; IP_SCORE(-2.05)[ip: (-8.11), ipnet: 2001:700::/32(-1.39), asn: 224(-0.75), country: NO(-0.01)] Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2020 13:21:52 -0000 On Tue, 3 Mar 2020 15:31+0300, Yuri Pankov wrote: > On 03.03.2020 15:27, Trond Endrestøl wrote: > > On Tue, 3 Mar 2020 12:56+0300, Yuri Pankov wrote: > > > > > On 03.03.2020 12:44, Trond Endrestøl wrote: > > > > On Tue, 3 Mar 2020 12:40+0300, Yuri Pankov wrote: > > > > > > > > > On 03.03.2020 11:49, Yuri Pankov wrote: > > > > > > With recent pkg fallout, I'm trying to build rust myself first time > > > > > > ever > > > > > > (as > > > > > > far as I can remember), and it's failing running out of swap on the > > > > > > following step: > > > > > > > > > > > > Building stage0 std artifacts (x86_64-unknown-freebsd -> > > > > > > x86_64-unknown-freebsd) > > > > > > running: > > > > > > "/usr/ports/lang/rust/work/rustc-1.41.1-src/build/x86_64-unknown-freebsd/stage0/bin/cargo" > > > > > > "build" "-Zconfig-profile" "--target" "x86_64-unknown-freebsd" > > > > > > "-Zbinary-dep-depinfo" "-j" "1" "-v" "--release" "--frozen" > > > > > > "--features" > > > > > > "panic-unwind backtrace compiler-builtins-c" "--manifest-path" > > > > > > "/usr/ports/lang/rust/work/rustc-1.41.1-src/src/libtest/Cargo.toml" > > > > > > "--message-format" "json-render-diagnostics" > > > > > > ^C^C^C > > > > > > Build completed unsuccessfully in 0:00:55 > > > > > > > > > > > > Here I pressed ^C as the build actually continues despite several > > > > > > rustdoc, > > > > > > python, and other processes being killed. > > > > > > > > > > > > swap_pager: out of swap space > > > > > > swp_pager_getswapspace(20): failed > > > > > > swap_pager: out of swap space > > > > > > swp_pager_getswapspace(11): failed > > > > > > > > > > > > The system has 32G of RAM and 2GB swap partition (as advised by > > > > > > zfs-auto > > > > > > installation option), top shows about 28G of memory free at that > > > > > > moment, > > > > > > so > > > > > > I'm wondering why the swap is being used, and if 2G should be enough > > > > > > to > > > > > > build rust. > > > > > > > > > > Looks like I got this wrong, adding a file-backed swap space I was > > > > > actually > > > > > able to run top, and seeing only 100M of memory being "Free", ~20G > > > > > memory > > > > > reported as "Active", and swap usage constantly growing being consumed > > > > > by > > > > > rustdoc process; something is really wrong here. > > > > > > > > Run top(1), hit the o key, type in size, and hit enter to have top > > > > sort the process list according to their virtual size. The culprit > > > > will eventually work its way to the top. > > > > > > Yes, it's rustdoc, and I'm seeing the same behavior as you described in > > > your > > > other reply. What's more interesting, having a little swap, processes get > > > killed almost immediately, and the build happily continues resulting > > > successful rust package. > > > > In my case, I see this behaviour twice during the build. I'll try and > > reduce my swap partition to 2 GiB and see if that makes a difference > > during the build. > > I wonder if you are seeing the problem for a long time, or it's something > recent for you? I just tried reinstalling the system from 20200227 snapshot, > and NOT seeing it, with or w/o the swap. Once I have everything installed, > I'll update to latest and re-check (note that I had WITH_CTF defined in > src.conf, though I doubt it's related). I have struggled for the past three weeks, with mad hypotheses ranging from ccache poisoning to faulty memory sticks, culminating in bootstrapping my localbase from scratch last weekend. One positive outcome is that I finally raised the size limit for my localbase ccache. -- Trond. From owner-freebsd-current@freebsd.org Tue Mar 3 14:11:54 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9498C250D30 for ; Tue, 3 Mar 2020 14:11:54 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 48WzTj4YxKz40gp for ; Tue, 3 Mar 2020 14:11:53 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 8ED4F250D2F; Tue, 3 Mar 2020 14:11:53 +0000 (UTC) Delivered-To: current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8D8A8250D2E for ; Tue, 3 Mar 2020 14:11:53 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48WzTf3VsFz40dd for ; Tue, 3 Mar 2020 14:11:49 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: by mail-wr1-f54.google.com with SMTP id v2so4444798wrp.12 for ; Tue, 03 Mar 2020 06:11:49 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=y1YIq9Ev/b4eqEWiBaoYRKX6tH6bTP3MzpVPMoPdn2w=; b=UyZLXm8QICLkmfh0FwSYBmZinAfoLmDU2ZXBB1c99IKC0WTRwixyzdhl+TzLK1IvvJ PgGvfEad5z4qEgCfxTAUOQNFNNy6bXPLu134VOxi9LXgNhVBknUS8GZkpfWQ2Q/9p7ge Mygii8sKMiOoHXyvBq7jzPJH0yAd/EHez1wGo5d37Erc9HjGy2iF8X+wZ1RBUPq95TqO 9tb9gshsAO8rzH492i94FMasqKwzXmV0X8Nd/IiEQpigeh0/93gfFw3pqfxaDauodtlP AN8EZpvvHXpY7xfhNkgLYzorK/YcDohBQs0hQNxwJkyBCEGQPtw7U2gJhcDQokG9cnZC wqIQ== X-Gm-Message-State: ANhLgQ2YQkgDwWPYFwzWRu5UwQQDG4ElJ00yuv11EUS2M/4XmhbUdCiO 3Tleng8KTxoy8fhFfZvAf8vuktmjsgI= X-Google-Smtp-Source: ADFU+vvLMj3lj6wwKn3WbV15fBNO6RgFH4Au57+hl+4cafS/CZgU8Mwu0HqbgpHUaxSxGEOwmf0inQ== X-Received: by 2002:adf:fc08:: with SMTP id i8mr5794140wrr.104.1583244707326; Tue, 03 Mar 2020 06:11:47 -0800 (PST) Received: from ?IPv6:2a02:8109:98c0:40ee:5e5f:67ff:fef4:ffd8? ([2a02:8109:98c0:40ee:5e5f:67ff:fef4:ffd8]) by smtp.gmail.com with ESMTPSA id g7sm30020314wrm.72.2020.03.03.06.11.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Mar 2020 06:11:46 -0800 (PST) Subject: Re: Wired memory consumes almost all my memory recently To: Kirill Ponomarev Cc: current@freebsd.org References: <6b9eba37-0866-fda6-4dd8-205eb3d77458@FreeBSD.org> <20200303104154.GB29722@krion.cc> From: Mateusz Piotrowski <0mp@FreeBSD.org> Message-ID: Date: Tue, 3 Mar 2020 15:12:09 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <20200303104154.GB29722@krion.cc> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Rspamd-Queue-Id: 48WzTf3VsFz40dd X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mpp302@gmail.com designates 209.85.221.54 as permitted sender) smtp.mailfrom=mpp302@gmail.com X-Spamd-Result: default: False [-3.53 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; DMARC_NA(0.00)[FreeBSD.org]; RCVD_COUNT_THREE(0.00)[3]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[54.221.85.209.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(-1.53)[ip: (-2.93), ipnet: 209.85.128.0/17(-2.99), asn: 15169(-1.66), country: US(-0.05)]; FORGED_SENDER(0.30)[0mp@FreeBSD.org,mpp302@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[54.221.85.209.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[0mp@FreeBSD.org,mpp302@gmail.com]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Mar 2020 14:11:54 -0000 On 3/3/20 11:41 AM, Kirill Ponomarev wrote: > On 03/03, Mateusz Piotrowski wrote: >> I'm experiencing some problems with memory on r358410. > Please try r358505, it should be fixed. Thanks! I'm running r358571 now. I'll post again if the problem persists.