Date: Tue, 3 Mar 2020 14:21:37 +0100 (CET) From: =?UTF-8?Q?Trond_Endrest=C3=B8l?= <trond.endrestol@ximalas.info> To: Yuri Pankov <ypankov@fastmail.com> Cc: current@freebsd.org Subject: Re: can't build rust -- out of swap space Message-ID: <alpine.BSF.2.22.395.2003031416220.6036@enterprise.ximalas.info> In-Reply-To: <993c159b-b9e0-cd47-5df6-73288748ae13@fastmail.com> References: <b86e3f65-fcbb-687f-ce94-a3415a561cd3@fastmail.com> <a6a9f0d2-5c06-9f77-73bb-ec51c0950b46@fastmail.com> <alpine.BSF.2.22.395.2003031042410.6036@enterprise.ximalas.info> <d8b08ab9-b925-aa74-ec8a-54477ab7beb9@fastmail.com> <alpine.BSF.2.22.395.2003031323360.6036@enterprise.ximalas.info> <993c159b-b9e0-cd47-5df6-73288748ae13@fastmail.com>
index | next in thread | previous in thread | raw e-mail
On Tue, 3 Mar 2020 15:31+0300, Yuri Pankov wrote: > On 03.03.2020 15:27, Trond Endrestøl wrote: > > On Tue, 3 Mar 2020 12:56+0300, Yuri Pankov wrote: > > > > > On 03.03.2020 12:44, Trond Endrestøl wrote: > > > > On Tue, 3 Mar 2020 12:40+0300, Yuri Pankov wrote: > > > > > > > > > On 03.03.2020 11:49, Yuri Pankov wrote: > > > > > > With recent pkg fallout, I'm trying to build rust myself first time > > > > > > ever > > > > > > (as > > > > > > far as I can remember), and it's failing running out of swap on the > > > > > > following step: > > > > > > > > > > > > Building stage0 std artifacts (x86_64-unknown-freebsd -> > > > > > > x86_64-unknown-freebsd) > > > > > > running: > > > > > > "/usr/ports/lang/rust/work/rustc-1.41.1-src/build/x86_64-unknown-freebsd/stage0/bin/cargo" > > > > > > "build" "-Zconfig-profile" "--target" "x86_64-unknown-freebsd" > > > > > > "-Zbinary-dep-depinfo" "-j" "1" "-v" "--release" "--frozen" > > > > > > "--features" > > > > > > "panic-unwind backtrace compiler-builtins-c" "--manifest-path" > > > > > > "/usr/ports/lang/rust/work/rustc-1.41.1-src/src/libtest/Cargo.toml" > > > > > > "--message-format" "json-render-diagnostics" > > > > > > ^C^C^C > > > > > > Build completed unsuccessfully in 0:00:55 > > > > > > > > > > > > Here I pressed ^C as the build actually continues despite several > > > > > > rustdoc, > > > > > > python, and other processes being killed. > > > > > > > > > > > > swap_pager: out of swap space > > > > > > swp_pager_getswapspace(20): failed > > > > > > swap_pager: out of swap space > > > > > > swp_pager_getswapspace(11): failed > > > > > > > > > > > > The system has 32G of RAM and 2GB swap partition (as advised by > > > > > > zfs-auto > > > > > > installation option), top shows about 28G of memory free at that > > > > > > moment, > > > > > > so > > > > > > I'm wondering why the swap is being used, and if 2G should be enough > > > > > > to > > > > > > build rust. > > > > > > > > > > Looks like I got this wrong, adding a file-backed swap space I was > > > > > actually > > > > > able to run top, and seeing only 100M of memory being "Free", ~20G > > > > > memory > > > > > reported as "Active", and swap usage constantly growing being consumed > > > > > by > > > > > rustdoc process; something is really wrong here. > > > > > > > > Run top(1), hit the o key, type in size, and hit enter to have top > > > > sort the process list according to their virtual size. The culprit > > > > will eventually work its way to the top. > > > > > > Yes, it's rustdoc, and I'm seeing the same behavior as you described in > > > your > > > other reply. What's more interesting, having a little swap, processes get > > > killed almost immediately, and the build happily continues resulting > > > successful rust package. > > > > In my case, I see this behaviour twice during the build. I'll try and > > reduce my swap partition to 2 GiB and see if that makes a difference > > during the build. > > I wonder if you are seeing the problem for a long time, or it's something > recent for you? I just tried reinstalling the system from 20200227 snapshot, > and NOT seeing it, with or w/o the swap. Once I have everything installed, > I'll update to latest and re-check (note that I had WITH_CTF defined in > src.conf, though I doubt it's related). I have struggled for the past three weeks, with mad hypotheses ranging from ccache poisoning to faulty memory sticks, culminating in bootstrapping my localbase from scratch last weekend. One positive outcome is that I finally raised the size limit for my localbase ccache. -- Trond. From owner-freebsd-current@freebsd.org Tue Mar 3 14:11:54 2020 Return-Path: <owner-freebsd-current@freebsd.org> Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9498C250D30 for <freebsd-current@mailman.nyi.freebsd.org>; Tue, 3 Mar 2020 14:11:54 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 48WzTj4YxKz40gp for <freebsd-current@freebsd.org>; Tue, 3 Mar 2020 14:11:53 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 8ED4F250D2F; Tue, 3 Mar 2020 14:11:53 +0000 (UTC) Delivered-To: current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8D8A8250D2E for <current@mailman.nyi.freebsd.org>; Tue, 3 Mar 2020 14:11:53 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48WzTf3VsFz40dd for <current@freebsd.org>; Tue, 3 Mar 2020 14:11:49 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: by mail-wr1-f54.google.com with SMTP id v2so4444798wrp.12 for <current@freebsd.org>; Tue, 03 Mar 2020 06:11:49 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=y1YIq9Ev/b4eqEWiBaoYRKX6tH6bTP3MzpVPMoPdn2w=; b=UyZLXm8QICLkmfh0FwSYBmZinAfoLmDU2ZXBB1c99IKC0WTRwixyzdhl+TzLK1IvvJ PgGvfEad5z4qEgCfxTAUOQNFNNy6bXPLu134VOxi9LXgNhVBknUS8GZkpfWQ2Q/9p7ge Mygii8sKMiOoHXyvBq7jzPJH0yAd/EHez1wGo5d37Erc9HjGy2iF8X+wZ1RBUPq95TqO 9tb9gshsAO8rzH492i94FMasqKwzXmV0X8Nd/IiEQpigeh0/93gfFw3pqfxaDauodtlP AN8EZpvvHXpY7xfhNkgLYzorK/YcDohBQs0hQNxwJkyBCEGQPtw7U2gJhcDQokG9cnZC wqIQ== X-Gm-Message-State: ANhLgQ2YQkgDwWPYFwzWRu5UwQQDG4ElJ00yuv11EUS2M/4XmhbUdCiO 3Tleng8KTxoy8fhFfZvAf8vuktmjsgI= X-Google-Smtp-Source: ADFU+vvLMj3lj6wwKn3WbV15fBNO6RgFH4Au57+hl+4cafS/CZgU8Mwu0HqbgpHUaxSxGEOwmf0inQ== X-Received: by 2002:adf:fc08:: with SMTP id i8mr5794140wrr.104.1583244707326; Tue, 03 Mar 2020 06:11:47 -0800 (PST) Received: from ?IPv6:2a02:8109:98c0:40ee:5e5f:67ff:fef4:ffd8? ([2a02:8109:98c0:40ee:5e5f:67ff:fef4:ffd8]) by smtp.gmail.com with ESMTPSA id g7sm30020314wrm.72.2020.03.03.06.11.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Mar 2020 06:11:46 -0800 (PST) Subject: Re: Wired memory consumes almost all my memory recently To: Kirill Ponomarev <kp@krion.cc> Cc: current@freebsd.org References: <6b9eba37-0866-fda6-4dd8-205eb3d77458@FreeBSD.org> <20200303104154.GB29722@krion.cc> From: Mateusz Piotrowski <0mp@FreeBSD.org> Message-ID: <cfdcb784-10c1-c810-72fc-49f27b2c5173@FreeBSD.org> Date: Tue, 3 Mar 2020 15:12:09 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <20200303104154.GB29722@krion.cc> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Rspamd-Queue-Id: 48WzTf3VsFz40dd X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mpp302@gmail.com designates 209.85.221.54 as permitted sender) smtp.mailfrom=mpp302@gmail.com X-Spamd-Result: default: False [-3.53 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; DMARC_NA(0.00)[FreeBSD.org]; RCVD_COUNT_THREE(0.00)[3]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[54.221.85.209.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(-1.53)[ip: (-2.93), ipnet: 209.85.128.0/17(-2.99), asn: 15169(-1.66), country: US(-0.05)]; FORGED_SENDER(0.30)[0mp@FreeBSD.org,mpp302@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[54.221.85.209.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[0mp@FreeBSD.org,mpp302@gmail.com]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current <freebsd-current.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>; List-Post: <mailto:freebsd-current@freebsd.org> List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=subscribe> X-List-Received-Date: Tue, 03 Mar 2020 14:11:54 -0000 On 3/3/20 11:41 AM, Kirill Ponomarev wrote: > On 03/03, Mateusz Piotrowski wrote: >> I'm experiencing some problems with memory on r358410. > Please try r358505, it should be fixed. Thanks! I'm running r358571 now. I'll post again if the problem persists.help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.22.395.2003031416220.6036>