From owner-freebsd-current@freebsd.org Tue Oct 30 12:23:30 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7584F10E2B6D for ; Tue, 30 Oct 2018 12:23:30 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1E6298245A; Tue, 30 Oct 2018 12:23:30 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id C7C4221BB7; Tue, 30 Oct 2018 12:23:29 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.195] (ptr-8rh08k1sdreej3vz59v.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:240e:402:f4d0:f72c:7583:2073]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id 4327F26417; Tue, 30 Oct 2018 13:23:27 +0100 (CET) From: "Kristof Provost" To: "Rodney W. Grimes" Cc: "Ernie Luzar" , "Bjoern A. Zeeb" , "FreeBSD current" Subject: Re: 12.0-BETA1 vnet with pf firewall Date: Tue, 30 Oct 2018 13:23:25 +0100 X-Mailer: MailMate (2.0BETAr6125) Message-ID: In-Reply-To: <7D8AB225-061D-4EEC-BC08-5B168F1B44E8@FreeBSD.org> References: <201810282139.w9SLdO58054096@pdx.rh.CN85.dnsmgr.net> <7D8AB225-061D-4EEC-BC08-5B168F1B44E8@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2018 12:23:30 -0000 On 29 Oct 2018, at 4:41, Kristof Provost wrote: > So we panic because we dereference a NULL pointer in strncmp(), which > happens because nprogtab = 13 but ef->progtab[12] has NULL pointers. > > It’s not clear to me why that happens, but it’s something to go > on. I do wonder if this isn’t a bit of a red herring too. It might > be an error in the error path (because we pass through > linker_file_unload()). link_elf_load_file() increments ef->nprogtab > for SHT_X86_64_UNWIND, so perhaps the error handling doesn’t cope > with that. > It looks like the root of the problem (failure to load) is in this line of link_elf_load_file(): ef->progtab[pb].addr = vnet_data_alloc(shdr[i].sh_size); The allocation of the vnet data fails. Bumping VNET_MODMIN in sys/net/vnet.c makes the load of ipfilter and pf succeed. I’m not too familiar with this part of the vnet code, but it looks to me like we’ve got more per-vnet variables that was originally anticipated, so we may need to just increase the allocated space. Best regards, Kristof From owner-freebsd-current@freebsd.org Tue Oct 30 13:30:07 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 352C410E8688 for ; Tue, 30 Oct 2018 13:30:07 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (cross.sbone.de [195.201.62.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B87018483E; Tue, 30 Oct 2018 13:30:06 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 2CE018D4A15C; Tue, 30 Oct 2018 13:29:57 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id D063BD1F857; Tue, 30 Oct 2018 13:29:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id C9Ehdd2dtM8X; Tue, 30 Oct 2018 13:29:54 +0000 (UTC) Received: from [192.168.1.88] (fresh-ayiya.sbone.de [IPv6:fde9:577b:c1a9:f001::2]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 9D1D4D1F856; Tue, 30 Oct 2018 13:29:53 +0000 (UTC) From: "Bjoern A. Zeeb" To: "Kristof Provost" Cc: "Rodney W. Grimes" , "Ernie Luzar" , "FreeBSD current" Subject: Re: 12.0-BETA1 vnet with pf firewall Date: Tue, 30 Oct 2018 13:29:52 +0000 X-Mailer: MailMate (2.0BETAr6125) Message-ID: <1B2DF00D-68FB-453F-82D0-6FC9C2BB6EE2@lists.zabbadoz.net> In-Reply-To: References: <201810282139.w9SLdO58054096@pdx.rh.CN85.dnsmgr.net> <7D8AB225-061D-4EEC-BC08-5B168F1B44E8@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2018 13:30:07 -0000 On 30 Oct 2018, at 12:23, Kristof Provost wrote: > On 29 Oct 2018, at 4:41, Kristof Provost wrote: >> So we panic because we dereference a NULL pointer in strncmp(), which >> happens because nprogtab = 13 but ef->progtab[12] has NULL pointers. I cannot reproduce your panic trying to load both pf, and ipfilter. I could only reproduce the original error. >> It’s not clear to me why that happens, but it’s something to go >> on. I do wonder if this isn’t a bit of a red herring too. It might >> be an error in the error path (because we pass through >> linker_file_unload()). link_elf_load_file() increments ef->nprogtab >> for SHT_X86_64_UNWIND, so perhaps the error handling doesn’t cope >> with that. >> > It looks like the root of the problem (failure to load) is in this > line of link_elf_load_file(): > > ef->progtab[pb].addr = > vnet_data_alloc(shdr[i].sh_size); > > The allocation of the vnet data fails. Bumping VNET_MODMIN in > sys/net/vnet.c makes the load of ipfilter and pf succeed. That’s interesting. I was thinking abut this but the error message (in case of not-panic) is only printed in case of ENOSYS: kern_linker.c: 486 /* 487 * If the file type has not been recognized by the last try 488 * printout a message before to fail. 489 */ 490 if (error == ENOSYS) 491 printf("%s: %s - unsupported file type\n", 492 __func__, filename); And I couldn’t see how we could get to that apart from one case in link_obj_elf.c: 578 if (hdr->e_type != ET_REL) { 579 error = ENOSYS; 580 goto out; 581 } And that’s a very unlikely one. So the ENOSYS must come out of somewhere else. I haven’t had enough time to track it down. We should. Could very well be some of the VM calls or something is returning that now and we just pass it back? > I’m not too familiar with this part of the vnet code, but it looks > to me like we’ve got more per-vnet variables that was originally > anticipated, so we may need to just increase the allocated space. Can you elfdump -a the two modules and see how big their set_vnet section sizes are? I see: pf.ko: sh_size: 6664 ipl.ko: sh_size: 2992 VNET_MODMIN is two pages (8k). So yes, that would exceed the module space. Having 6.6k global variable space is a bit excessive? Where does that come from? multicast used to have a similar problem in the past that it could not be loaded as a module as it had a massive array there and we changed it to be malloced and that reduced it to a pointer. 0000000000000f38 l O set_vnet 0000000000000428 vnet_entry_pfr_nulltable 0000000000000b10 l O set_vnet 00000000000003d0 vnet_entry_pf_default_rule 0000000000001370 l O set_vnet 0000000000000690 vnet_entry_pf_main_anchor 0000000000000000 l O set_vnet 0000000000000120 vnet_entry_pf_status seem to be the biggies. /bz