From nobody Sat Jun 15 02:23:53 2024 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4W1KhV2gjCz5N2hx for ; Sat, 15 Jun 2024 02:23:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4W1KhV1Y4Hz43tM for ; Sat, 15 Jun 2024 02:23:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1718418234; a=rsa-sha256; cv=none; b=OKEK5qJTHLoFhzIBfPd5ceeJQhrltdtGGpaGBe2e7cz5Uia5uPAxC2iJOBEp+fcGIPEsOA 6KvnvffCJ1TZpLJxvMdWjHQ9yQ8YKBP6Z1jfAQxE8NDMjPEW26wtvDUacXSlux0zhsXw8V sbpPUlTnKeGu+WbLGntR+ZhMAw7XPyQp3y6PkBYuQbr9oyOPKiLEgaCo5vMmRGEBbb7AFr KjAeXXQWSbaRLgDn97gjgte+DF8YWGW2joH63fMUpRmcAagXvnMqwgofIfgG3i86WVR7lu as7q3+y79NZpFkdpwu3dzYv9x3GwFZJLriIegmxbvI2hn/TTn4h+mA6bTqiSlg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1718418234; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=tcyTxfQ/i6t4C5gKr5BOnYqEG5nyTdoIdY+a43+8A/s=; b=Vs5dU075PM2I4uDOZKdYjk2alCgE16aF5FVmQtgghJGDAH9Q9tQfxNQydFgKx6ppgUFPOl GQWZTo8s1cjQ+fqlS33VPWiNgNht1InXgNNDnLND09lgZ5xeECUjx4tDmcFCbQmJTD39HB u1ppBDd+PAIVnDQbEBaVUVyT323Ddw9HvVCAYV0sO/GtmiqAKin03NKO0qXrQEu1/hqJod 4vbaZyYPgWmLzc3QH8FzULA8KwIZvkpnujWpJVa9/gAy6NmI9jYhBrTKib60oDxlpLTcby PKBpAQb+9UJLK0yHjYgfgef6cIqmFVAnq++g0f2ydW+ikCz65M68Uh7xItTlDA== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4W1KhV0pDLzlnF for ; Sat, 15 Jun 2024 02:23:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 45F2Nsdw057383 for ; Sat, 15 Jun 2024 02:23:54 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 45F2Nsbn057382 for bugs@FreeBSD.org; Sat, 15 Jun 2024 02:23:54 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 279742] 14.1-RELEASE hangs compiling pspp requiring reboot Date: Sat, 15 Jun 2024 02:23:53 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: dgilbert@eicat.ca X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter attachments.created Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D279742 Bug ID: 279742 Summary: 14.1-RELEASE hangs compiling pspp requiring reboot Product: Base System Version: 14.0-STABLE Hardware: amd64 OS: Any Status: New Severity: Affects Many People Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: dgilbert@eicat.ca Created attachment 251458 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D251458&action= =3Dedit core.txt of crash. I clicked on 14.0-STABLE because 14.1-RELEASE was not yet a choice. I upgraded my poudriere box to 14.1, created a new jail for 14.1, and launc= hed into a "-a" build pretty much immediately after returning from BSDCan. The build machine is a Threadripper 1900X with 128G of RAM and 140TB of disk in RAID-Z2. It has stably built poudriere almost constantly since I upgraded = it to it's current state --- about 3 years or so. After the first poudriere hang, I instrumented things like temperatures. N= one of these spiked, but the hang happened again and again. After awhile, it w= as clear that pspp compiling was the trigger. Note that pspp would have compi= led under 14.0 less than a week before (ie: just before BSDCan). I had to get debugging in to my kernel and learn how to cause it to debug.= =20 That took a couple tries --- all-the-while repeatedly crashing while pspp w= as building. Top was up on the window I keep open ... and this was the last t= op on display. last pid: 31372; load averages: 21.72, 32.46, 41.6670=20=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 up 0+04:34:59 20:36:48 220 processes: 12 running, 192 sleeping, 2 zombie, 14 waiting CPU: 21.7% user, 0.0% nice, 40.4% system, 0.0% interrupt, 37.8% idle Mem: 32M Active, 264K Inact, 124G Wired, 604M Free ARC: 16G Total, 230M MFU, 334M MRU, 22M Anon, 15G Header, 191M Other 107M Compressed, 460M Uncompressed, 4.28:1 Ratio Swap: 256G Total, 98G Used, 158G Free, 38% Inuse, 2868K In, 3612K Out PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMM= AND 61759 root 2 166 i10 60M 2616K vofflo 3 40:43 75.20% pspp-output 6367 root 2 166 i10 88M 2604K vofflo 15 36:06 73.63% pspp-output 15409 root 2 166 i10 92M 2608K vofflo 2 33:53 72.64% pspp-output 81893 root 2 166 i10 86M 2600K CPU12 12 34:05 72.04% pspp-output 78622 root 2 166 i10 57M 2588K CPU11 11 28:42 69.19% pspp-output 25531 root 2 166 i10 95M 2616K CPU5 5 27:00 68.84% pspp-output 81789 root 2 166 i10 42M 2584K CPU6 6 23:16 65.11% pspp-output 87988 root 2 166 i10 102M 2596K CPU7 7 20:57 64.28% pspp-output 11364 root 2 166 i10 57M 2612K CPU10 10 19:50 64.14% pspp-output 23538 root 2 166 i10 66M 2604K CPU11 11 21:09 63.94% pspp-output 61379 root 2 166 i10 93M 2624K tmpfs 4 21:10 63.46% pspp-output 85836 root 2 166 i10 74M 2608K CPU14 14 19:19 62.69% pspp-output 58400 root 2 166 i10 76M 2440K RUN 5 13:26 56.27% pspp-output 58294 root 2 166 i10 72M 2444K CPU1 1 14:44 56.15% pspp-output 70050 root 2 166 i10 48M 2440K RUN 1 12:46 56.10% pspp-output 2561 root 1 20 0 303M 1728K select 12 1:09 0.40% smbd 2502 postgres 1 20 0 173M 1012K select 4 0:13 0.16% post= gres 65067 root 1 20 0 17M 1452K CPU9 9 0:21 0.14% top 2577 root 1 20 0 17M 1216K select 9 0:40 0.13% tmux 72517 root 6 166 i10 2310M 452K uwait 1 9:40 0.07% ghc-9.6.4 8903 root 45 166 i10 34G 4716K uwait 0 12:30 0.06% java 2503 postgres 1 20 0 31M 684K select 7 0:08 0.05% post= gres 37351 root 1 20 0 22M 328K select 9 0:03 0.05% sshd 2190 root 1 20 0 14M 172K select 6 0:00 0.03% sysl= ogd 72294 root 11 166 i10 345M 1664K kqread 4 0:02 0.01% node 2294 root 1 20 0 280M 228K select 11 0:00 0.01% httpd 1192 root 1 20 0 18M 340K select 0 0:27 0.01% moun= td 1162 ntpd 1 20 0 23M 520K select 12 0:01 0.01% ntpd 95259 root 1 20 0 12M 328K ttyin 4 0:03 0.01% cu 1749 uwsgi 1 20 0 57M 412K kqread 12 0:01 0.00% uwsgi-3.8 36420 root 1 20 0 19M 544K select 5 0:01 0.00% mini= com 1307 root 1 20 0 164M 460K kqread 8 0:00 0.00% php-= fpm 1253 root 128 68 0 12M 2316K rpcsvc 11 0:13 0.00% nfsd 91926 root 2 166 i10 74M 2908K pfault 15 123:07 0.00% pspp-output 72530 root 11 166 i10 7498M 836K pfault 5 99:32 0.00% node 46100 root 18 166 i10 261G 932K uwait 4 18:33 0.00% dotn= et 73028 root 1 166 i10 165M 4096B WAIT 11 3:56 0.00% 2955 root 1 166 i10 15M 4096B wait 13 3:24 0.00% 93083 root 1 166 i10 195M 4096B WAIT 13 3:14 0.00% 22537 root 6 166 i10 298M 17M uwait 10 1:22 0.00% ld.l= ld 2588 root 1 166 i10 22M 224K select 6 1:02 0.00% sh 24257 root 1 166 i10 145M 4096B WAIT 12 0:42 0.00% 1301 www 1 20 0 27M 4096B WAIT 5 0:32 0.00% 90301 root 14 166 i10 261G 260K uwait 4 0:24 0.00% dotn= et It's worth noting here that the virtual terminal switch (alt - F) works after this happens, but no other input is recognized (can't hit return in a window and shells going through the machine to others don't continue their output). When it happened this time, I dropped to KDB and dumped. core.txt attached. NOTE: this is repeatable. I have been through the cycle 6 times so far. --=20 You are receiving this mail because: You are the assignee for the bug.=