From nobody Fri Jul 14 13:35:45 2023 X-Original-To: questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4R2XYH5JpXz4nDFH for ; Fri, 14 Jul 2023 13:35:47 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smarthost1.sentex.ca", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4R2XYH2bS6z4Dfb for ; Fri, 14 Jul 2023 13:35:47 +0000 (UTC) (envelope-from mike@sentex.net) Authentication-Results: mx1.freebsd.org; none Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [199.212.134.19]) by smarthost1.sentex.ca (8.17.1/8.16.1) with ESMTPS id 36EDZjWf046057 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=FAIL); Fri, 14 Jul 2023 09:35:46 -0400 (EDT) (envelope-from mike@sentex.net) Received: from [IPV6:2607:f3e0:0:4::29] ([IPv6:2607:f3e0:0:4:0:0:0:29]) by pyroxene2a.sentex.ca (8.16.1/8.15.2) with ESMTPS id 36EDZjPf031814 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Fri, 14 Jul 2023 09:35:45 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: Date: Fri, 14 Jul 2023 09:35:45 -0400 List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: Random sshd hangs Content-Language: en-US To: Doug Hardie , questions@freebsd.org References: <30FBD752-56E4-4A18-8EB9-54E9F4D0EF98@sermon-archive.info> From: mike tancsa In-Reply-To: <30FBD752-56E4-4A18-8EB9-54E9F4D0EF98@sermon-archive.info> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.84 X-Rspamd-Queue-Id: 4R2XYH2bS6z4Dfb X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On 7/13/2023 11:14 PM, Doug Hardie wrote: > After somewhere around 5-10 minutes, everything returns to normal. >  This is not consistent. I can run the program 10 time fine before the > next hang.  There is nothing I have found that is common to the hangs. >  Now that I have the program running correctly, the system never seems > to hang again.  All my ideas on what is causing this are not supported > by the facts.  It's not a core dump issue.  It does not appear to be > an IP issue as IP continues to work, as do new ssh connections.  The > web server is sitting there completely idle.  It never receives any > requests.  I suppose it could be something in libgd, but it would seem > that would be continuing even now.  I am a lot confused over this.  I > don't want to upgrade my production servers as Those hangs would be > killers to my clients.  Any ideas what this might be, or what to > investigate if it happens again? > Maybe login on a new session and then start with something simple like truss to attach to one of the hung processes and see what its doing and where it is ? Perhaps have a pcap running too on the outbound interface to see if the server processes are trying to send data out and its really the client thats locking up somehow ? (do an "echo hello | wall " and see if data gets sent on all the sessions).  Increasing the verbosity of the sshd config to debug1 might shed some light too so you can see what the server process thinks is happening.  Does your app write to a pty ? maybe it was hitting other pty sessions by accident ?     ---Mike