From nobody Mon May 19 04:46:33 2025 X-Original-To: sysinstall@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4b14s55LpYz5wTFS for ; Mon, 19 May 2025 04:46:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4b14s52s29z44BJ for ; Mon, 19 May 2025 04:46:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1747629993; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8BFIG6W+B68h1aP8+xIUNYfyeWZ28wN751dTvvbZko8=; b=D5+blW+TMXLvDMWOSy2OV5nAEJfSXeu6Jl4/5fN6ty4F7wK17sJpm80pXm0sYOrwySbqUJ egbekvuVypuKD+slOtrK1MYbw78P/P2QAHR1Os89UUWu2UkbhMgaoNgfPLR/Of+ZmS4/4W IMSBi0WCs4DuZOAFKxuhwkQX1pnagPBqncdwCfs7OeYUJVvs0uQN9uXU1D6Gp0amXIBso2 dcZ2PDg3imBLY2t+oz1eOZNxcXLmdorUIpIj6yt0RdkL8dMAmY8Wt2fOQ0qsTfDCqT8zaY vuLzrmzZMZYn4PS7SqL1r8znWF2RJPAU6T9oulzGDV7r4gKydPtDWjZwBWPxLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1747629993; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8BFIG6W+B68h1aP8+xIUNYfyeWZ28wN751dTvvbZko8=; b=csv9oZV//y9axZEDUtjuAfzgvx9uTA+L0ddtDMrfN+BVUC35PUNXiPikrbzcXN7OVyLzGS 70YhoVBIpuMfIxQXGaDKK65pCRVbDh5v6kz9js0HA+Neu5WU8KXI3r9KG1Yfpki0nirYwl 0QB1D7a+rtbZQiFhsZmIp3Rbyveup/9503XoQ07oZQAtFiPoAKqnNMWBcXPMccOv5bPzuT YanoVtFElzSSxpqhQHBmk06huaSySf1pXWD9MVHyBzo+ywm0YjUa+ylqwCjoystv5Y9hkW lmzA+ATs9ca4FiUyAcEMp9hzriPmB38MhtqzZcjSr0n5VYOe4Rg8BDuxsuYTgQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1747629993; a=rsa-sha256; cv=none; b=pfIdClzsEpne0rJNCeu5dC7fMOFjm43Xa6RbgZ/QHy7iKWBsXA2MKefab/9BnAm5LODzYl BEmKqHq7G5CJVcr6CHuD/KPXleG0icYvAXrOMDHcYM0vBK/OkKEL4DJrAaVBX9LGqz1E4e XJqCJOkOymz7eLn3fzNKRMSqZkwH2Puab+5kLeRgvEpOaT9V71AkvnE071Som+ZW0g25fh o7AQmCkhVGQ+5aGbcLOiU4hUwv12mL6Nc9zeuaKtjkgfIH8wOZ/rS4fZJCN9LjK/tkS1hb RcivoQz/fpEHnNqxyM+MnIu7XCp9Mb/hlTk9fIxbe/Ysqm/e2msd+sWvPDeGjg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4b14s51tR7zqHw for ; Mon, 19 May 2025 04:46:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 54J4kXHA096313 for ; Mon, 19 May 2025 04:46:33 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 54J4kXd2096312 for sysinstall@FreeBSD.org; Mon, 19 May 2025 04:46:33 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: sysinstall@FreeBSD.org Subject: [Bug 286703] During bsdinstall, pkg hangs outputting to serial console Date: Mon, 19 May 2025 04:46:33 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rpokala@panasas.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: sysinstall@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Sysinstall Work List-Archive: https://lists.freebsd.org/archives/freebsd-sysinstall List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-sysinstall@freebsd.org Sender: owner-freebsd-sysinstall@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D286703 --- Comment #4 from Ravi Pokala --- Some additional observations: 1. Apart from `bsdinstall', I found that one of our internal utilities would consistently cause the serial console to go non-responsive. The same utility run via the KVM works without issue. Interestingly, redirecting the output = of the utility to a file and then `cat'ing it on the serial console, does *not* cause the console to go non-responsive. 2. If running that utility on the serial console causes it to go non-responsive, breaking into the debugger (NMI or video console) causes it= to emit the output from the utility to the console, before it brings up the debugger. This means that characters directed to the serial console were continuing to be buffered, but the buffer was not actually being flushed to= the console. My colleagues and I agree that (2) in particular has the feel of a missed interrupt. And then, serendipity! While we were debugging this problem, @cpercival committed: https://cgit.freebsd.org/src/commit/?id=3D9a685c09f06a which disabled a workaround from a dozen years ago: https://cgit.freebsd.org/src/commit/?id=3D1c60b24baa50 and a more recent related change https://cgit.freebsd.org/src/commit/?id=3D7e7f7beee732 That is talking about an emulated UART from QEMU and Xen and EC2, but it se= ems to describe our issue with the physical hardware: TX FIFO didn't interrupt while emptying, stopping further flushing. And when something force the flu= sh to happen -- NMI via BMC, or video console debugger sequence -- the buffer finishes flushing. We added 'hw.broken_txfifo=3D"1"' to /boot/loader.conf to enable the workar= ound, and that solved the problem. We completed several dozen successful installations with `bsdinstall', and as many runs of the utility which previously triggered the hang. We need to do some testing to see if the fix causes latency spikes on real hardware the way it did on EC2, which is why @cpercival disabled it there. --=20 You are receiving this mail because: You are the assignee for the bug.=