From owner-freebsd-current@freebsd.org Mon Oct 5 18:12:37 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B8D2742BE55 for ; Mon, 5 Oct 2020 18:12:37 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x844.google.com (mail-qt1-x844.google.com [IPv6:2607:f8b0:4864:20::844]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C4pbn1my5z4JQZ for ; Mon, 5 Oct 2020 18:12:36 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x844.google.com with SMTP id m9so9748401qth.7 for ; Mon, 05 Oct 2020 11:12:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=s6rXRvaqObCBE+PcUF2wuBkXcH4yr4B9+otQ9Lm2Jeg=; b=MhfTOPHOHExfUJEk4gGJ/JOgG/HhpyHjWFoB7rsVdttT1D+nKTvsvo31hXfnTSqbRO DnWkdFJjROWhQB40JYy9QDKFFxEVVvgwX8c413zTv8RmbHyW1nFvuZ8gmPDFk89YnGbw 8Ymb6hVr9W1jS+wpVyKQJoamA62hQN33J8FFiwMsoYASStIAyk/HwyNtJB4YvVIDzOvR PrBgGT9KZf7Dof9oxAuHEvV5gWjhI1HoyIEqoYXn70j6baeUqIHh49zYhOrKVueSYn+I rfHK42/qJMcJE6EJLcIPzN53//xwL+rTqFLM45C31xxaabOJ3emq5v5PEXhRBuExC2Gh P1xA== X-Gm-Message-State: AOAM531KxBxLkK0Vujs6dQVLYjq/3MAjZDy27Xi4l8jAh72xp4G9AFSg P9+hbY+PwjpjGXPtNvsbosEvVYJ8qtKIzd+KjCBPyQ== X-Google-Smtp-Source: ABdhPJxF21m+VolnGvGmFYjGbBwINcYozl9Ms/V2oriEhsfU4mad+rVB3IePEwoniDHfaSTHqQvgoD3XKcA2gJq00Sg= X-Received: by 2002:ac8:31af:: with SMTP id h44mr1112723qte.175.1601921555821; Mon, 05 Oct 2020 11:12:35 -0700 (PDT) MIME-Version: 1.0 References: <1dec1b9d-03e7-7171-9f48-71573dafead4@delphij.net> <95d6ce61-06af-eb12-9a6e-2b8342663dc7@delphij.net> In-Reply-To: <95d6ce61-06af-eb12-9a6e-2b8342663dc7@delphij.net> From: Warner Losh Date: Mon, 5 Oct 2020 12:12:25 -0600 Message-ID: Subject: Re: GPF on boot with devmatch To: Xin LI Cc: FreeBSD Current , Warner Losh X-Rspamd-Queue-Id: 4C4pbn1my5z4JQZ X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.20 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-0.88)[-0.882]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.02)[-1.019]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; MIME_BASE64_TEXT(0.10)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::844:from]; NEURAL_HAM_SHORT(-0.40)[-0.396]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; MAILMAN_DEST(0.00)[freebsd-current] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Oct 2020 18:12:37 -0000 On Sun, Oct 4, 2020 at 11:30 PM Xin Li wrote: > > > On 10/4/20 10:13 PM, Warner Losh wrote: > > > > > > On Sun, Oct 4, 2020, 11:07 PM Xin Li > > wrote: > > > > Hi, > > > > I'm seeing this panic at boot after upgrading from r366217 to > r366364, > > and continues to exist for r366421 (but I haven't find out the exact > > change that caused it). Preloading the relevant kernel modules > > (uhid.ko, ums.ko and wmt.ko) seems to make the kernel boot correctly. > > > > > > What happens if you disable devmatch and load these modules by hand? > > Loading these modules from loader or kld_list will prevent this panic > regardless if devmatch is enabled. > > > What happens if you load them from rc.d scripts with devmatch disabled? > > It seems that the devmatch was started by devd and not rc. Disabling > devmatch (setting devmatch_enable="NO" without loading any of these klds > would not provoke the panic). > That sounds like a bug to me... I'll have to look into it. I'm mostly asking 'what happens if you load them after boot' to see if this is a 'devmatch doing weird things' issue or a 'these drivers don't like to be loaded so late and we need to track down why' issue. Warner > > Warner > > > > This is not reproducible on my laptop, which will load many more > kernel > > modules. > > > > === > > Autoloading module: uhid.ko > > Autoloading module: wmt.ko > > > > > > Fatal trap 9: general protection fault while in kernel mode > > cpuid = 2; apic id = 04 > > instruction pointer = 0x20:0xffffffff806ad6eb > > stack pointer = 0x28:0xfffffe01850cd960 > > frame pointer = 0x28:0xfffffe01850cd9e0 > > code segment = base 0x0, limit 0xfffff, type 0x1b > > = DPL 0, pres 1, long 1, def32 0, gran 1 > > processor eflags = interrupt enabled, resume, IOPL = 0 > > current process = 740 (devmatch) > > trap number = 9 > > panic: general protection fault > > cpuid = 3 > > time = 1601866799 > > KDB: stack backtrace: > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > > 0xfffffe01850cd670 > > vpanic() at vpanic+0x182/frame 0xfffffe01850cd6c0 > > panic() at panic+0x43/frame 0xfffffe01850cd720 > > trap_fatal() at trap_fatal+0x387/frame 0xfffffe01850cd780 > > trap() at trap+0xa4/frame 0xfffffe01850cd890 > > calltrap() at calltrap+0x8/frame 0xfffffe01850cd890 > > --- trap 0x9, rip = 0xffffffff806ad6eb, rsp = 0xfffffe01850cd960, > rbp = > > 0xfffffe01850cd9e0 --- > > sysctl_devices() at sysctl_devices+0x24b/frame 0xfffffe01850cd9e0 > > sysctl_root_handler_locked() at sysctl_root_handler_locked+0x9c/frame > > 0xfffffe01850cda30 > > sysctl_root() at sysctl_root+0x20a/frame 0xfffffe01850cdab0 > > userland_sysctl() at userland_sysctl+0x17d/frame 0xfffffe01850cdb60 > > sys___sysctl() at sys___sysctl+0x5f/frame 0xfffffe01850cdc10 > > amd64_syscall() at amd64_syscall+0x135/frame 0xfffffe01850cdd30 > > fast_syscall_common() at fast_syscall_common+0xf8/frame > > 0xfffffe01850cdd30 > > --- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x80038968a, > rsp = > > 0x7fffffffd988, rbp = 0x7fffffffd9c0 --- > > === > > > > sysctl_devices+0x24b (0x6dab) was: > > > > sb->s_len += strlen(p); > > 6d50: 4c 89 e7 mov %r12,%rdi > > 6d53: e8 00 00 00 00 callq 6d58 > > > > 6d58: 48 01 45 b0 add %rax,-0x50(%rbp) > > 6d5c: 48 8d 7d 88 lea -0x78(%rbp),%rdi > > sbuf_putc(&sb, '\0'); > > 6d60: 31 f6 xor %esi,%esi > > 6d62: e8 00 00 00 00 callq 6d67 > > > > MPASS((sb->s_flags & SBUF_INCLUDENUL) == 0); > > 6d67: f6 45 b8 02 testb $0x2,-0x48(%rbp) > > 6d6b: 0f 85 10 01 00 00 jne 6e81 > > > > if (sb->s_error != 0) > > 6d71: 83 7d a0 00 cmpl $0x0,-0x60(%rbp) > > 6d75: 0f 85 8c 00 00 00 jne 6e07 > > > > p = EOB(sb); > > 6d7b: 4c 8b 65 88 mov -0x78(%rbp),%r12 > > 6d7f: 48 8b 45 b0 mov -0x50(%rbp),%rax > > *p = '\0'; /* sbuf buffer isn't NUL terminated until > > sbuf_finish() */ > > 6d83: 41 c6 04 04 00 movb $0x0,(%r12,%rax,1) > > space = SPACE(sb); > > 6d88: 4c 8b 6d a8 mov -0x58(%rbp),%r13 > > 6d8c: 4c 2b 6d b0 sub -0x50(%rbp),%r13 > > if (space <= 1) { > > 6d90: 49 83 fd 01 cmp $0x1,%r13 > > 6d94: 77 09 ja 6d9f > > > > sb->s_error = ENOMEM; > > 6d96: c7 45 a0 0c 00 00 00 movl $0xc,-0x60(%rbp) > > 6d9d: eb 68 jmp 6e07 > > > > 6d9f: 49 01 c4 add %rax,%r12 > > return (dev->parent); > > 6da2: 48 8b 7b 28 mov 0x28(%rbx),%rdi > > if (parent == NULL) { > > 6da6: 48 85 ff test %rdi,%rdi > > 6da9: 74 4b je 6df6 > > > > KOBJOPLOOKUP(((kobj_t)_dev)->ops,bus_child_location_str); > > 6dab: 48 8b 07 mov (%rdi),%rax > > 6dae: 48 c7 c2 00 00 00 00 mov $0x0,%rdx > > 6db5: 0f b6 0d 00 00 00 00 movzbl 0x0(%rip),%ecx > # > > 6dbc > > 6dbc: 4c 8b 04 c8 mov (%rax,%rcx,8),%r8 > > 6dc0: 49 39 10 cmp %rdx,(%r8) > > 6dc3: 74 22 je 6de7 > > > > 6dc5: 48 8d 34 c8 lea (%rax,%rcx,8),%rsi > > 6dc9: 48 89 7d d0 mov %rdi,-0x30(%rbp) > > 6dcd: 48 8b b8 00 08 00 00 mov 0x800(%rax),%rdi > > 6dd4: 48 c7 c2 00 00 00 00 mov $0x0,%rdx > > 6ddb: e8 00 00 00 00 callq 6de0 > > > > 6de0: 48 8b 7d d0 mov -0x30(%rbp),%rdi > > 6de4: 49 89 c0 mov %rax,%r8 > > rc = ((bus_child_location_str_t *) _m)(_dev, _child, _buf, > > _buflen); > > 6de7: 48 89 de mov %rbx,%rsi > > >