From owner-freebsd-current@freebsd.org Sat May 12 15:02:16 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 00AC2FDFF6E for ; Sat, 12 May 2018 15:02:16 +0000 (UTC) (envelope-from se@freebsd.org) Received: from mailout09.t-online.de (mailout09.t-online.de [194.25.134.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailout00.t-online.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 77A9B7FE37 for ; Sat, 12 May 2018 15:02:15 +0000 (UTC) (envelope-from se@freebsd.org) Received: from fwd36.aul.t-online.de (fwd36.aul.t-online.de [172.20.26.137]) by mailout09.t-online.de (Postfix) with SMTP id CFAA34256EC9 for ; Sat, 12 May 2018 16:54:51 +0200 (CEST) Received: from Stefans-MBP-10.fritz.box (Z6NtQgZOZhY-3zfsNM8loQ7+pW6F4tg7Qa+UidsDxYB5Uu01N7iXzwtsR+beDx9grJ@[84.154.105.176]) by fwd36.t-online.de with (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384 encrypted) esmtp id 1fHVv5-1XcLOS0; Sat, 12 May 2018 16:54:47 +0200 To: FreeBSD Current From: Stefan Esser Subject: Strange problem with tar in -CURRENT (VM problem?) Openpgp: preference=signencrypt Autocrypt: addr=se@freebsd.org; prefer-encrypt=mutual; keydata= xsBNBFVxiRIBCADOLNOZBsqlplHUQ3tG782FNtVT33rQli9EjNt2fhFERHIo4NxHlWBpHLnU b0s4L/eItx7au0i7Gegv01A9LUMwOnAc9EFAm4EW3Wmoa6MYrcP7xDClohg/Y69f7SNpEs3x YATBy+L6NzWZbJjZXD4vqPgZSDuMcLU7BEdJf0f+6h1BJPnGuwHpsSdnnMrZeIM8xQ8PPUVQ L0GZkVojHgNUngJH6e21qDrud0BkdiBcij0M3TCP4GQrJ/YMdurfc8mhueLpwGR2U1W8TYB7 4UY+NLw0McThOCLCxXflIeF/Y7jSB0zxzvb/H3LWkodUTkV57yX9IbUAGA5RKRg9zsUtABEB AAHNKVN0ZWZhbiBFw59lciAoWWFob28hKSA8c3QuZXNzZXJAeWFob28uZGU+wsCWBBMBCgBA AhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AWIQSjceplnAvsyCtxUxNH67XvWv31RAUC WvLvqwUJCyUBEwAKCRBH67XvWv31REySCACc6vqcSFQCRyBRc2CV5ZBjbbnTy7VBoXbUS3/c 4Hn8I0YQ39q7//2z8vYsgLeM1mMXL4PUIU/0f0dBAFBLpxV7bntGzyCJls6SeGS/qcQKhqaI 6I7NcWg8OkIJIhUL6q238cS1ql9pU65fyHe0PP8JS08m81PDpX2/4wTE6h2jgYUy55eXRzoF MEjr1S8SSnidsBem27o7iWu9ltJsUtE86071iZlLzbuHv2nvucrjAV9cK9tHrxYT/YiY8QhT L48iWj2xIjLjg1ebmgIFZ2k881we/KTIoUugqOOR1gDSc4qwM8CA388cN3frjtl98CwhAT5T UV8tIDqri+/Z1AKwzsBNBFVxiRIBCACxI/aglzGVbnI6XHd0MTP05VK/fJub4hHdc+LQpz1M kVnCAhFbY9oecTB/togdKtfiloavjbFrb0nJhJnx57K+3SdSuu+znaQ4SlWiZOtXnkbpRWNU eMm+gtTDMSvloGAfr76RtFHskdDOLgXsHD70bKuMhlBxUCrSwGzHaD00q8iQPhJZ5itb3WPq z3B4IjiDAWTO2obD1wtAvSuHuUj/XJRsiKDKW3x13cfavkad81bZW4cpNwUv8XHLv/vaZPSA ly+hkY7NrDZydMMXVNQ7AJQufWuTJ0q7sImRcEZ5EIa98esJPey4O7C0vY405wjeyxpVZkpq ThDMurqtQFn1ABEBAAHCwHwEGAEKACYCGwwWIQSjceplnAvsyCtxUxNH67XvWv31RAUCWvLv qwUJCyUBGQAKCRBH67XvWv31RLnrB/9gzcRlpx71sDMosoZULWn7wysBJ/8AIEfIByRaHQe3 pn/KwE57pB+zFbbQqB7YzeZb7/UUgR4zU2ZbOcEfwDZcHUbj0B3fGRsS3t0uiLlAd8w0sBZb SxrqzjdpDjIbOZkxssqUmvrsN67UG1AFWH9aD24keBS7YjPBS8hLxPeYV+Xz6vUL8fRZje/Z JgiBMIwyj6g2lH/zkdnxBdC0iG1xxJOLTaghMMeQyCdH6ef8+VMyAlAJsMckbOTvx63tY8z7 DFcrnTJfbe1EziRilVsEaK8tTzJzhcTfos+f3eBYWEilxe5HzIhYKJeC7lmsSUcGwa6+9VRg a0ctmi9Z8OgX Message-ID: <19a18e0f-d881-cecd-3bae-4145fac10be4@freebsd.org> Date: Sat, 12 May 2018 16:54:46 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit X-ID: Z6NtQgZOZhY-3zfsNM8loQ7+pW6F4tg7Qa+UidsDxYB5Uu01N7iXzwtsR+beDx9grJ X-TOI-MSGID: 62171993-49cf-46d4-93e0-52235dbef88f X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 May 2018 15:02:16 -0000 While searching for the reason an upgrade of math/atlas failed on my amd64 -CURRENT system, I found that tar fails to create an archive of some 10KB. It is killed (-9) after some 30 seconds during which it grows seemingly without bounds. The port processes some TAR files in order to fixup paths in them with the following shell loop (edited for readability): cd ${WRKDIR}/ATLAS/CONFIG/ARCHS for t in *.tgz ; do /bin/mv ${t} ${t}.bak /usr/bin/tar -s '/gcc/gcc6/' -xf ${t}.bak /usr/bin/tar -czf ${t} ${t%.tgz} # (***) /bin/rm -f -r ${t%.tgz} ${t}.bak done The command that fails is the one marked (***) and I have tried to trace it with ktrace and truss, but only see that a large amount of memory is mapped and the tar process is killed without having produced any output. I have added "-v" to watch progress and the log does also indicate, that tar does not even start to write to the archive. Removal of the "z" option makes no difference. Typical "ps l" output is: UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 2269 2254 0 30 0 105946804 21244044 pfault D 0 0:31,48 /usr/bin/tar -czf Core232SSE3.tgz Core232SSE3 (bsdtar) VSZ is 105946804 KB or about 100 GB, RSS 21 GB when tar is killed ... The files to be processed are: -rw-r--r-- 1 root wheel 11399 May 12 16:40 AMD64K10h32SSE3.tgz -rw-r--r-- 1 root wheel 11697 May 12 16:40 AMD64K10h64SSE3.tgz -rw-r--r-- 1 root wheel 1305 May 12 16:40 BOZOL1.tgz -rw-r--r-- 1 root wheel 9909 May 12 16:40 Core232SSE3.tgz drwxr-xr-x 5 root wheel 9 Feb 25 2009 Core264SSE3/ -rw-r--r-- 1 root wheel 0 May 12 16:40 Core264SSE3.tgz -rw-r--r-- 1 root wheel 10212 May 14 2011 Core264SSE3.tgz.bak -rw-r--r-- 1 root wheel 8544 May 14 2011 Corei164SSE3.tgz [...] The failure may be caused by a race-condition, since sometimes tar fails on a later file (e.g. Corei164SSE3.tgz). If I replace "${TAR} -czf" with "gtar -czf", then the port can be built. But I do not think that this is a problem in BSDTAR, since the failure can be reproduced (also after a buildworld/buildkernel and reboot), but there have been no changes to BSDTAR since the libarcjive upgrade in January. I guess this is a VM problem, that happens to show itself in this specific program invocation. (The system runs without other obvious problems and tar works outside this specific usage in the port ...) Any ideas? Best regards, STefan