Date: Tue, 3 Mar 2020 13:27:34 +0100 (CET) From: =?UTF-8?Q?Trond_Endrest=C3=B8l?= <trond.endrestol@ximalas.info> To: Yuri Pankov <ypankov@fastmail.com> Cc: current@freebsd.org Subject: Re: can't build rust -- out of swap space Message-ID: <alpine.BSF.2.22.395.2003031323360.6036@enterprise.ximalas.info> In-Reply-To: <d8b08ab9-b925-aa74-ec8a-54477ab7beb9@fastmail.com> References: <b86e3f65-fcbb-687f-ce94-a3415a561cd3@fastmail.com> <a6a9f0d2-5c06-9f77-73bb-ec51c0950b46@fastmail.com> <alpine.BSF.2.22.395.2003031042410.6036@enterprise.ximalas.info> <d8b08ab9-b925-aa74-ec8a-54477ab7beb9@fastmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 3 Mar 2020 12:56+0300, Yuri Pankov wrote: > On 03.03.2020 12:44, Trond Endrestøl wrote: > > On Tue, 3 Mar 2020 12:40+0300, Yuri Pankov wrote: > > > > > On 03.03.2020 11:49, Yuri Pankov wrote: > > > > With recent pkg fallout, I'm trying to build rust myself first time ever > > > > (as > > > > far as I can remember), and it's failing running out of swap on the > > > > following step: > > > > > > > > Building stage0 std artifacts (x86_64-unknown-freebsd -> > > > > x86_64-unknown-freebsd) > > > > running: > > > > "/usr/ports/lang/rust/work/rustc-1.41.1-src/build/x86_64-unknown-freebsd/stage0/bin/cargo" > > > > "build" "-Zconfig-profile" "--target" "x86_64-unknown-freebsd" > > > > "-Zbinary-dep-depinfo" "-j" "1" "-v" "--release" "--frozen" "--features" > > > > "panic-unwind backtrace compiler-builtins-c" "--manifest-path" > > > > "/usr/ports/lang/rust/work/rustc-1.41.1-src/src/libtest/Cargo.toml" > > > > "--message-format" "json-render-diagnostics" > > > > ^C^C^C > > > > Build completed unsuccessfully in 0:00:55 > > > > > > > > Here I pressed ^C as the build actually continues despite several > > > > rustdoc, > > > > python, and other processes being killed. > > > > > > > > swap_pager: out of swap space > > > > swp_pager_getswapspace(20): failed > > > > swap_pager: out of swap space > > > > swp_pager_getswapspace(11): failed > > > > > > > > The system has 32G of RAM and 2GB swap partition (as advised by zfs-auto > > > > installation option), top shows about 28G of memory free at that moment, > > > > so > > > > I'm wondering why the swap is being used, and if 2G should be enough to > > > > build rust. > > > > > > Looks like I got this wrong, adding a file-backed swap space I was > > > actually > > > able to run top, and seeing only 100M of memory being "Free", ~20G memory > > > reported as "Active", and swap usage constantly growing being consumed by > > > rustdoc process; something is really wrong here. > > > > Run top(1), hit the o key, type in size, and hit enter to have top > > sort the process list according to their virtual size. The culprit > > will eventually work its way to the top. > > Yes, it's rustdoc, and I'm seeing the same behavior as you described in your > other reply. What's more interesting, having a little swap, processes get > killed almost immediately, and the build happily continues resulting > successful rust package. In my case, I see this behaviour twice during the build. I'll try and reduce my swap partition to 2 GiB and see if that makes a difference during the build. -- Trond. From owner-freebsd-current@freebsd.org Tue Mar 3 12:31:56 2020 Return-Path: <owner-freebsd-current@freebsd.org> Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2579E26E303 for <freebsd-current@mailman.nyi.freebsd.org>; Tue, 3 Mar 2020 12:31:56 +0000 (UTC) (envelope-from ypankov@fastmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 48WxGM2YSXz3CY0 for <freebsd-current@freebsd.org>; Tue, 3 Mar 2020 12:31:55 +0000 (UTC) (envelope-from ypankov@fastmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 48EED26E302; Tue, 3 Mar 2020 12:31:55 +0000 (UTC) Delivered-To: current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4837B26E301 for <current@mailman.nyi.freebsd.org>; Tue, 3 Mar 2020 12:31:55 +0000 (UTC) (envelope-from ypankov@fastmail.com) Received: from wnew3-smtp.messagingengine.com (wnew3-smtp.messagingengine.com [64.147.123.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48WxGK4jf1z3CTH for <current@freebsd.org>; Tue, 3 Mar 2020 12:31:53 +0000 (UTC) (envelope-from ypankov@fastmail.com) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.west.internal (Postfix) with ESMTP id 6D779542; Tue, 3 Mar 2020 07:31:51 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Tue, 03 Mar 2020 07:31:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h= subject:to:cc:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm2; bh=3 pvCAL7lfMK7CoVwriBh8FIJrt51s5/yhh9mwZwx+6k=; b=JLRyVYR4o8yN2Cm4Y uWsyHFixRaoabKNeerWnSeBQsHcfmVAaT3UXHYbff7kdiOsrzYe+1llCGHYKGgqo Vvpk4z//E4yXOAdb9lZiUSQbWiR4/QcvkqVOPTlsGE6R3EX/JwDl9bJ0JM8Y2Svl MB50Jw99z6dlS0PxfJ4o6fZWkthSaB+nP3uHifWojsdXaQID4aYKJG13Qa6Xfe4q yJCrgrv0CSifFnAvO+WJumk1rL9jMSeypb7DviO+rutxVzpbq78XUfNiWABr4eRt z0tiXF05YSNwE6yCX1kPHoiEsTyxSFRw+agXQC+so0WSThLnhqzpDXAsTPIr9CYO vjHew== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=3pvCAL7lfMK7CoVwriBh8FIJrt51s5/yhh9mwZwx+ 6k=; b=IILFjLy6IwjPGnYEVUlT5z1EQ4wZiAEWJHTE5mHS+qbNmM6zOO6VIPFih mDeYGJ8MiPTwzj3N3ln6zGkP7NkW8iH5VJC93tM6bQoqAtdZWbos2CWlfPnUseZ1 fRXLyMvNW0f7RTHf47wRTHdX7MUJFfi+rC+ske8Lw2VhjmimV3XKtVI3zjmEZy4+ yfiuy8ON0eitRbzBifdGss51f9I+0O+zX2wIhhXEHP9XPYALomIp22qAT0ckxTwv +MgxTVRFcP6FKDXIyLp3a3p46HpyjXs/U7U+wvbmNg+FRLznE7r5Cm0nnfcvGyos AGk3WecTwjDCyupgRqzgRMKTqaD5g== X-ME-Sender: <xms:Nk5eXrM1Kc-f3VUDnwfLO0lZqK_wk8ZB_SzgnYrMkcGxbw2-vH7y4g> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedugedruddtiedggeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefuvfhfhffkffgfgggjtgfgsehtke ertddtfeejnecuhfhrohhmpegjuhhrihcurfgrnhhkohhvuceohihprghnkhhovhesfhgr shhtmhgrihhlrdgtohhmqeenucfkphephedrudefledrudeltddrudekieenucevlhhush htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeihphgrnhhkohhvsehf rghsthhmrghilhdrtghomh X-ME-Proxy: <xmx:Nk5eXhEYQjpL4BvatxfUfz7kHL9NfCFA8RyGAcmZLCTQjZx1QTMf_w> <xmx:Nk5eXgosHbDlCdxAYgl8gJGF-kCMvDjEsK_q9qKtm0BHhyqrJh0K7A> <xmx:Nk5eXqqsx7lf-jxlYrffsDWksOQmXZ_6FQFGl7NVy6SWNxttRoW6ow> <xmx:N05eXmHB2Rud2a6A0WNkEnwObuBPAQf_SWd5k75J1YXK8ROnkiS2-UvOhDs> Received: from earth.lan (unknown [5.139.190.186]) by mail.messagingengine.com (Postfix) with ESMTPA id E16363061393; Tue, 3 Mar 2020 07:31:49 -0500 (EST) Subject: Re: can't build rust -- out of swap space To: =?UTF-8?Q?Trond_Endrest=c3=b8l?= <trond.endrestol@ximalas.info> Cc: current@freebsd.org References: <b86e3f65-fcbb-687f-ce94-a3415a561cd3@fastmail.com> <a6a9f0d2-5c06-9f77-73bb-ec51c0950b46@fastmail.com> <alpine.BSF.2.22.395.2003031042410.6036@enterprise.ximalas.info> <d8b08ab9-b925-aa74-ec8a-54477ab7beb9@fastmail.com> <alpine.BSF.2.22.395.2003031323360.6036@enterprise.ximalas.info> From: Yuri Pankov <ypankov@fastmail.com> Message-ID: <993c159b-b9e0-cd47-5df6-73288748ae13@fastmail.com> Date: Tue, 3 Mar 2020 15:31:48 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <alpine.BSF.2.22.395.2003031323360.6036@enterprise.ximalas.info> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 48WxGK4jf1z3CTH X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=fastmail.com header.s=fm2 header.b=JLRyVYR4; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=IILFjLy6; dmarc=pass (policy=none) header.from=fastmail.com; spf=pass (mx1.freebsd.org: domain of ypankov@fastmail.com designates 64.147.123.17 as permitted sender) smtp.mailfrom=ypankov@fastmail.com X-Spamd-Result: default: False [-3.10 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[fastmail.com:s=fm2,messagingengine.com:s=fm2]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.17]; FREEMAIL_FROM(0.00)[fastmail.com]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[fastmail.com:+,messagingengine.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[fastmail.com,none]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[17.123.147.64.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[fastmail.com]; ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.74), ipnet: 64.147.123.0/24(-4.92), asn: 11403(-2.69), country: US(-0.05)]; RECEIVED_SPAMHAUS_PBL(0.00)[186.190.139.5.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current <freebsd-current.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>; List-Post: <mailto:freebsd-current@freebsd.org> List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=subscribe> X-List-Received-Date: Tue, 03 Mar 2020 12:31:56 -0000 On 03.03.2020 15:27, Trond Endrestøl wrote: > On Tue, 3 Mar 2020 12:56+0300, Yuri Pankov wrote: > >> On 03.03.2020 12:44, Trond Endrestøl wrote: >>> On Tue, 3 Mar 2020 12:40+0300, Yuri Pankov wrote: >>> >>>> On 03.03.2020 11:49, Yuri Pankov wrote: >>>>> With recent pkg fallout, I'm trying to build rust myself first time ever >>>>> (as >>>>> far as I can remember), and it's failing running out of swap on the >>>>> following step: >>>>> >>>>> Building stage0 std artifacts (x86_64-unknown-freebsd -> >>>>> x86_64-unknown-freebsd) >>>>> running: >>>>> "/usr/ports/lang/rust/work/rustc-1.41.1-src/build/x86_64-unknown-freebsd/stage0/bin/cargo" >>>>> "build" "-Zconfig-profile" "--target" "x86_64-unknown-freebsd" >>>>> "-Zbinary-dep-depinfo" "-j" "1" "-v" "--release" "--frozen" "--features" >>>>> "panic-unwind backtrace compiler-builtins-c" "--manifest-path" >>>>> "/usr/ports/lang/rust/work/rustc-1.41.1-src/src/libtest/Cargo.toml" >>>>> "--message-format" "json-render-diagnostics" >>>>> ^C^C^C >>>>> Build completed unsuccessfully in 0:00:55 >>>>> >>>>> Here I pressed ^C as the build actually continues despite several >>>>> rustdoc, >>>>> python, and other processes being killed. >>>>> >>>>> swap_pager: out of swap space >>>>> swp_pager_getswapspace(20): failed >>>>> swap_pager: out of swap space >>>>> swp_pager_getswapspace(11): failed >>>>> >>>>> The system has 32G of RAM and 2GB swap partition (as advised by zfs-auto >>>>> installation option), top shows about 28G of memory free at that moment, >>>>> so >>>>> I'm wondering why the swap is being used, and if 2G should be enough to >>>>> build rust. >>>> >>>> Looks like I got this wrong, adding a file-backed swap space I was >>>> actually >>>> able to run top, and seeing only 100M of memory being "Free", ~20G memory >>>> reported as "Active", and swap usage constantly growing being consumed by >>>> rustdoc process; something is really wrong here. >>> >>> Run top(1), hit the o key, type in size, and hit enter to have top >>> sort the process list according to their virtual size. The culprit >>> will eventually work its way to the top. >> >> Yes, it's rustdoc, and I'm seeing the same behavior as you described in your >> other reply. What's more interesting, having a little swap, processes get >> killed almost immediately, and the build happily continues resulting >> successful rust package. > > In my case, I see this behaviour twice during the build. I'll try and > reduce my swap partition to 2 GiB and see if that makes a difference > during the build. I wonder if you are seeing the problem for a long time, or it's something recent for you? I just tried reinstalling the system from 20200227 snapshot, and NOT seeing it, with or w/o the swap. Once I have everything installed, I'll update to latest and re-check (note that I had WITH_CTF defined in src.conf, though I doubt it's related).
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.22.395.2003031323360.6036>