From nobody Wed Oct 13 14:46:29 2021 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 17B251805717 for ; Wed, 13 Oct 2021 14:46:34 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qt1-x82c.google.com (mail-qt1-x82c.google.com [IPv6:2607:f8b0:4864:20::82c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HTwMs6phFz3syy for ; Wed, 13 Oct 2021 14:46:33 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qt1-x82c.google.com with SMTP id z24so2712089qtv.9 for ; Wed, 13 Oct 2021 07:46:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=Ny6CwDbROpuZPpBCPMdEZQKH7OHfqwBYFlmIiv1forE=; b=mVJye9+R+qUKRNdwyWiiTrsjbXlYaeLMqre4OrPMfcz8kzwmv37R6Dj+EccM/bJQQ+ JqosYp/igeMZMyzFB9U/0Ea8RH+1dqJcP+8u0mF0jqoh35vBo0dMJkAkL36PB9nHmRCw A9JIIosgOGUp/pLFhYzNbSLmvrIIquxXo8dSCpIJWD6F50O/HDGC3KWV6ji4doS4ibL3 iWQDKDFQP1wT77gYbOd+NmeVcaCUM/h2QogotlCnPzdjIK8Vy+XYNIRjr2HiPfdfRKCL jaZSeyNvh1NBXFFdMr5Ad/17imor/8MTv1MQxA1bfaCCQ9v0fHuoTINjsvcg2qD36eoH P8jA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition :content-transfer-encoding:in-reply-to; bh=Ny6CwDbROpuZPpBCPMdEZQKH7OHfqwBYFlmIiv1forE=; b=qz9doORnYlwcBiKDe1na2AWPzVsNWg7878LaMR8ncV80FZ/2Q0snSZRfQCA4ZNQb9t RLLQSDCh3oU6fEmejBhuo+6i52PuSsCczJvVECYd+3WnFrvmuTP5i7IpZfKGAtxAOV8k C0d4Pgrh6Q6fRUA/Ucs4P8wzETwNgZaY09DA+1KUJ5ifbluPFRx19V4qXk2WQngvtlSy xeckWhqm1F3DLJICdEBWcSesJzVEY+TV8yUeML0Oj8BBe5CXFd39xaX4z9tyG8qTXsIS 1MpeRvx3xE2DElnZSuoEZZwfR1qnAXldq8iWaEXALJvCEGttcL7ARDJg4ZY/ySbq/5Oc T/eQ== X-Gm-Message-State: AOAM5319Xn9/l15HA0bMrudTTogY57QMR6lMcvE5CRTYhlB5J1Rx6Y5S ZgG5R2aMKchU+xThIgbh7Bk= X-Google-Smtp-Source: ABdhPJwtw1iercHpBwIQvbuGOjd0EP7w3DNgw3SbuN+DKRE17zCn0ol2QH7vZcRGFvjiaP9XgDCLbQ== X-Received: by 2002:ac8:7319:: with SMTP id x25mr29920982qto.147.1634136393383; Wed, 13 Oct 2021 07:46:33 -0700 (PDT) Received: from nuc ([142.126.186.191]) by smtp.gmail.com with ESMTPSA id t11sm6039660qkm.92.2021.10.13.07.46.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Oct 2021 07:46:31 -0700 (PDT) Date: Wed, 13 Oct 2021 10:46:29 -0400 From: Mark Johnston To: Peter Eriksson Cc: freebsd-current Subject: Re: Panic in pipe_write from syslogd in 12.2? Message-ID: References: <0FFAABE0-F260-4FE0-8B1D-1F179C502170@lysator.liu.se> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <0FFAABE0-F260-4FE0-8B1D-1F179C502170@lysator.liu.se> X-Rspamd-Queue-Id: 4HTwMs6phFz3syy X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Tue, Oct 12, 2021 at 08:39:55PM +0200, Peter Eriksson wrote: > I just noticed that a couple of my 12.2-RELEASE-p4 running servers have… 8263, 14474 and 3831 defunct subprocesses from syslogd and also seems to have stopped writing to the log files… When I tried to kill syslogd on a fourth server (with some X000 defunct processes) the machine panic’ed and rebooted. > > I seem to have a vague memory of this being a known bug/someone saw something similar or perhaps even solved in later patch releases? But my google-fu seems to be failing me today. Anyone else remember? I don't believe we've released a patch that would fix this. The unix domain socket code has been refactored a fair bit with respect to locking since 12.2, and I believe this panic will be fixed in 12.3. In particular, I've seen one other report of a similar panic that went away after https://cgit.freebsd.org/src/commit/?id=ccdadf1a9bb64156e4a62bb6207c37b841467cb7 . > (The one that panic’ed is now running -p10 instead which they should have done a long time ago but…) > > > I reported it on the FreeBSD bugzilla: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259084 > > > Output from one that still is running: > # egrep syslogd /var/log/sys/15:10/procstat-kk-a.log > 9212 101640 syslogd - mi_switch+0xd4 sleepq_catch_signals+0x403 sleepq_wait_sig+0xf _sleep+0x1de pipe_write+0x583 dofilewrite+0xb0 sys_write+0xc0 amd64_syscall+0x387 fast_syscall_common+0xf8 > > Output from the one that panic’ed: > Fatal trap 12: page fault while in kernel mode > cpuid = 20; apic id = 14 > fault virtual address = 0x410 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff80b9f55c > stack pointer = 0x28:0xfffffe14debc6710 > frame pointer = 0x28:0xfffffe14debc6790 > code segment = base r x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 9277 (sshd) > trap number = 12 > panic: page fault > cpuid = 20 > time = 1633990484 > KDB: stack backtrace: > #0 0xffffffff80c0ad75 at kdb_backtrace+0x65 > #1 0xffffffff80bbf02b at vpanic+0x17b > #2 0xffffffff80bbeea3 at panic+0x43 > #3 0xffffffff8108e911 at trap_fatal+0x391 > #4 0xffffffff8108e96f at trap_pfault+0x4f > #5 0xffffffff8108dfb6 at trap+0x286 > #6 0xffffffff81066c28 at calltrap+0x8 > #7 0xffffffff80c6365f at unp_pcb_owned_lock2_slowpath+0x12f > #8 0xffffffff80c61e0f at uipc_send+0x139f > #9 0xffffffff80c55b7a at sosend_generic+0x4ca > #10 0xffffffff80c55f90 at sosend+0x50 > #11 0xffffffff80c5cc55 at kern_sendit+0x225 > #12 0xffffffff80c5cfcc at sendit+0x19c > #13 0xffffffff80c5ce1d at sys_sendto+0x4d > #14 0xffffffff8108f4c7 at amd64_syscall+0x387 > #15 0xffffffff8106754e at fast_syscall_common+0xf8 > Uptime: 212d21h35m47s > > - Peter >