From nobody Tue Oct 19 08:35:51 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4A912180A9D6 for ; Tue, 19 Oct 2021 08:35:55 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HYRsR1h4mz3lhw; Tue, 19 Oct 2021 08:35:55 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.not-for.work (onlyone.not-for.work [IPv6:2a01:4f8:201:6350::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: lev/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 015002B615; Tue, 19 Oct 2021 08:35:54 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [192.168.134.16] (unknown [94.19.243.255]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id 73B904F; Tue, 19 Oct 2021 11:35:52 +0300 (MSK) Message-ID: <2362acb7-4286-d2d4-ece4-56df13db7ed3@FreeBSD.org> Date: Tue, 19 Oct 2021 11:35:51 +0300 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.2.0 Reply-To: lev@FreeBSD.org Subject: Re: Does audit work on stable/12? Audit-related panic on latest stable/12 Content-Language: en-US To: Alan Somers Cc: freebsd-stable References: From: Lev Serebryakov Organization: FreeBSD In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-ThisMailContainsUnwantedMimeParts: N On 19.10.2021 7:36, Alan Somers wrote: >> I've upgraded one of my servers from 11.4 to latest stable/12. This server is unique in me fleet because it has audit (and auditd) enabled. >> >> First of all, right after (source-based, buildworld & Ko) upgrade dmesg becomes flooded with: >> >> BSM conversion requested for unknown event 43224 >> BSM conversion requested for unknown event 43225 >> BSM conversion requested for unknown event 43234 >> BSM conversion requested for unknown event 43238 >> >> And after several minutes of work I've got panic: >> >> Sleeping thread (tid 101199, pid 51147) owns a non-sleepable lock >> BSM conversion requested for unknown event 43224 >> KDB: stack backtrace of thread 101199: >> #0 0xffffffff804d0f34 at mi_switch+0xd4 >> BSM conversion requested for unknown event 43224 >> BSM conversion requested for unknown event 43224 >> #1 0xffffffff8051ca2c at sleepq_wait+0x2c >> #2 0xffffffff80467d62 at _cv_wait+0xf2 >> #3 0xffffffff80719573 at audit_commit+0x243 >> #4 0xffffffff80719866 at audit_syscall_exit+0x26 >> #5 0xffffffff804d7f8a at kern_thr_exit+0x14a >> #6 0xffffffff804d7e37 at sys_thr_exit+0x67 >> #7 0xffffffff807a1557 at amd64_syscall+0x387 >> #8 0xffffffff8077a7ae at fast_syscall_common+0xf8 >> panic: sleeping thread >> cpuid = 6 >> time = 1634604615 >> KDB: stack backtrace: >> #0 0xffffffff8050e925 at kdb_backtrace+0x65 >> #1 0xffffffff804c5bcb at vpanic+0x17b >> #2 0xffffffff804c5a43 at panic+0x43 >> #3 0xffffffff80523702 at propagate_priority+0x282 >> #4 0xffffffff805242cc at turnstile_wait+0x30c >> #5 0xffffffff804abd29 at __mtx_lock_sleep+0x199 >> #6 0xffffffff804d7ec2 at kern_thr_exit+0x82 >> #7 0xffffffff804d7e37 at sys_thr_exit+0x67 >> #8 0xffffffff807a1557 at amd64_syscall+0x387 >> #9 0xffffffff8077a7ae at fast_syscall_common+0xf8 >> >> >> Now, I've turned off auditd and server looks Ok (at least, it is stable for 30 minutes). But I need audit on this server. Is it known problem? Is it configuration problem? > > audit has at least some coverage in CI, but apparently not enough. > Would you share your /etc/security configuration? Event 43224 is /etc/security was merged from my old (stable/11 era) config by `mergemaster`. Here result is: audit_control: # # $FreeBSD$ # host: dir:/var/audit minfree:5 dist:off flags:lo,aa,fc,-fd,fw,pc,nt,ex naflags:lo,aa,fc,-fd,fw,pc,nt,ex policy:cnt,argv filesz:200M expire-after:356d OR 50G audit_user: # # $FreeBSD$ # root:lo:no daemon::+fw,+fc,+fd operator::+fw,+fc,+fd bin::+fw,+fc,+fd tty::+fw,+fc,+fd kmem::+fw,+fc,+fd games::+fw,+fc,+fd news::+fw,+fc,+fd man::+fw,+fc,+fd sshd::+fw,+fc,+fd smmsp::+fw,+fc,+fd mailnull::+fw,+fc,+fd bind::+fw,+fc,+fd proxy::+fw,+fc,+fd _pflogd::+fw,+fc,+fd _dhcp::+fw,+fc,+fd uucp::+fw,+fc,+fd pop::+fw,+fc,+fd www::+fw,+fc,+fd hast::+fw,+fc,+fd nobody::+fw,+fc,+fd mysql::+fw,+fc,+fd postfix::+fw,+fc,+fd dovecot::+fw,+fc,+fd dovenull::+fw,+fc,+fd All other audit_* files are identical with source ones. > thr_new, which certainly should be known everywhere, so I'm wondering > if you have a bad build somehow. Are you using GENERIC or do you have > a custom kernel config? It is custom (trimmed) kernel config. Nothing special, only most of devices (which is not actual on this hardware) are stripped. -- // Lev Serebryakov