Date: Thu, 4 May 2023 18:11:06 +0800 From: Zhenlei Huang <zlei@FreeBSD.org> To: Konstantin Belousov <kostikbel@gmail.com> Cc: Hans Petter Selasky <hselasky@freebsd.org>, FreeBSD CURRENT <freebsd-current@freebsd.org>, Gleb Smirnoff <glebius@freebsd.org> Subject: Re: Link modules to DYN type Message-ID: <B2F70E53-BC01-4E13-925F-589CF1A9B5B7@FreeBSD.org> In-Reply-To: <86DC0C8E-56E8-49B9-8441-D64C690FA5F0@FreeBSD.org> References: <97390FE1-1DF5-43A1-A3F4-2B945D681437@FreeBSD.org> <2bb66cac-c7f1-e45b-693a-8afbda05cfa6@freebsd.org> <ZEkHKJ_BRhV22gf_@kib.kiev.ua> <86DC0C8E-56E8-49B9-8441-D64C690FA5F0@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_41DA2938-02A5-4B92-9856-2D3CD7B51E9C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Apr 28, 2023, at 12:32 AM, Zhenlei Huang <zlei@FreeBSD.org> wrote: >=20 >=20 >=20 >> On Apr 26, 2023, at 7:12 PM, Konstantin Belousov <kostikbel@gmail.com = <mailto:kostikbel@gmail.com>> wrote: >>=20 >> On Wed, Apr 26, 2023 at 12:55:02PM +0200, Hans Petter Selasky wrote: >>> On 4/26/23 12:36, Zhenlei Huang wrote: >>>> Hi, >>>>=20 >>>> I'm recently working on https://reviews.freebsd.org/D39638 = <https://reviews.freebsd.org/D39638>; (sysctl(9): Enable vnet sysctl = variables be loader tunable), >>>> the changes to `sys/kern/link_elf_obj.c` are runtime tested, but = not those to `sys/kern/link_elf.c` . >>>>=20 >>>> After some hacking I realized that `link_elf.c` is for EXEC = (Executable file) or DYN (Shared object file), and `link_elf_obj.c` is >>>> for REL (Relocatable file). >>>>=20 >>>> ``` >>>> /* link_elf.c */ >>>> static int >>>> link_elf_load_file(linker_class_t cls, const char* filename, >>>> linker_file_t* result) >>>> { >>>> ... >>>> if (hdr->e_type !=3D ET_EXEC && hdr->e_type !=3D ET_DYN) { >>>> error =3D ENOSYS; >>>> goto out; >>>> } >>>> ... >>>> } >>>>=20 >>>>=20 >>>> /* link_elf_obj.c */ >>>> static int >>>> link_elf_load_file(linker_class_t cls, const char *filename, >>>> linker_file_t *result) >>>> { >>>> ... >>>> if (hdr->e_type !=3D ET_REL) { >>>> error =3D ENOSYS; >>>> goto out; >>>> } >>>> ... >>>> } >>>> ``` >>>>=20 >>>> Run the following snip: >>>> ``` >>>> # find /boot/kernel -type f -name "*.ko" -exec readelf -h {} \; | = grep Type >>>> ``` >>>> shows that all the kernel modules' types are `REL (Relocatable = file)`. >>>>=20 >>>> I guess if some module such as if_bridge is linked to DYN type, = then I can do runtime for the changes to `sys/kern/link_elf.c`. >>>>=20 >>>> I'm not familiar with elf and linkers, is that ( compile module and = link it to DYN type ) possible ? >>=20 >> Module file type (shared object vs. object file) depends on = architecture. >> For amd64 modules are objects, while kernel is shared library. >> For arm64 (and all other arches, I believe) modules and kernels are = shared >> libraries. >>=20 >> I think you can link amd64 module as shared object, but this require = enough >> hacking of the build infrastructure. At least I am not aware of a = simple >> knob to switch the produced type. >=20 > I did some hack on `sys/conf/kmod.mk` and finally produced DYN kernel = modules. > The good news is I do some basic sysctl tests, but the bad news is the = module does not function correctly. >=20 > For exampe the if_disc.c >=20 > ``` > static void > vnet_disc_init(const void *unused __unused) > { > /* Reference V_disc_cloner will immediately trigger page fault = panic */ > V_disc_cloner =3D if_clone_simple(discname, disc_clone_create, > disc_clone_destroy, 0); > } > VNET_SYSINIT(vnet_disc_init, SI_SUB_PSEUDO, SI_ORDER_ANY, > vnet_disc_init, NULL); > ``` >=20 > I suspect the relocation is not done correctly for DYN elf kmod on = amd64. >=20 > My local patch to kmod.mk: >=20 > ``` > diff --git a/sys/conf/kmod.mk b/sys/conf/kmod.mk > index 134b150af1d9..1fc5386204a5 100644 > --- a/sys/conf/kmod.mk > +++ b/sys/conf/kmod.mk > @@ -84,6 +84,7 @@ __KLD_SHARED=3Dyes > .else > __KLD_SHARED=3Dno > .endif > +__KLD_SHARED=3Dyes > =20 > .if !empty(CFLAGS:M-O[23s]) && empty(CFLAGS:M-fno-strict-aliasing) > CFLAGS+=3D -fno-strict-aliasing > @@ -167,6 +168,7 @@ CFLAGS+=3D -fno-omit-frame-pointer = -mno-omit-leaf-frame-pointer > ${MACHINE_CPUARCH} =3D=3D "powerpc" > CFLAGS+=3D -fPIC > .endif > +CFLAGS+=3D -fPIC > =20 > # Temporary workaround for PR 196407, which contains the fascinating = details. > # Don't allow clang to use fpu instructions or registers in kernel = modules. > ``` >=20 >=20 > As for https://reviews.freebsd.org/D39638 = <https://reviews.freebsd.org/D39638>, for other platform such as arm, I = think the `link_elf_propagate_vnets()` should work if `parse_vnet()` = works. >=20 > I'll appreciate if someone can test it on platforms those have DYN = type kernel modules.=20 Good news on this! I've managed to test the change for `link_elf.c` on QEMU RISCV . It = works as expected and solid ! >=20 >>=20 >>=20 >>>>=20 >>>=20 >>> Hi, >>>=20 >>> I don't have an answer for you either, but I have seen in the past, = loading >>> kernel modules behaves a bit like libraries, in the following = regard: >>>=20 >>> If two kernel modules define the same global symbol, then no warning = is >>> given and the first loaded symbol definition (I think) is used to = resolve >>> that symbol for all kernel modules, regardless of the prototype. = Probably we >>> should not allow this. That's why building LINT is a good thing, to = avoid >>> this issue. >> No, in-kernel linker does not behave this way. >> Modules need to contain explicit reference to all modules they depend = upon, >> using the MODULE_DEPEND() macro. Only symbols from the dependencies = are >> resolved. >>=20 >> All modules get an implicit reference to kernel. >>=20 >>>=20 >>> Even if we don't have C++ support in the FreeBSD kernel, defining = symbol >>> names the way C++ does for C could be nice for the kernel too, also = with >>> regards to debugging systems. >>>=20 >>> Many times when I don't know what is going on, I do like this: >>>=20 >>> #include <sys/kdb.h> >>>=20 >>> .... >>>=20 >>> if (not too fast or my sysctl debug) { >>> printf("My tracer\n"); >>> kdb_backtrace(); >>> } >>>=20 >>> Dtrace can also do this, but not during boot. Just track who is = calling >>> those functions, and you'll probably find the answer to your = question! >>>=20 >>> --HPS >>>=20 >>>>=20 >>>> Best regards, >>>> Zhenlei --Apple-Mail=_41DA2938-02A5-4B92-9856-2D3CD7B51E9C Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br = class=3D""><div><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D"">On Apr 28, 2023, at 12:32 AM, Zhenlei Huang <<a = href=3D"mailto:zlei@FreeBSD.org" class=3D"">zlei@FreeBSD.org</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><meta = charset=3D"UTF-8" class=3D""><br class=3D"Apple-interchange-newline"><br = class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 13px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><blockquote type=3D"cite" class=3D"" style=3D"font-family: = Helvetica; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; orphans: auto; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; = -webkit-text-stroke-width: 0px; text-decoration: none;"><div class=3D"">On= Apr 26, 2023, at 7:12 PM, Konstantin Belousov <<a = href=3D"mailto:kostikbel@gmail.com" class=3D"">kostikbel@gmail.com</a>>= wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><span= class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: = Menlo-Regular; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none; float: none; display: inline !important;">On Wed, Apr 26, 2023 at = 12:55:02PM +0200, Hans Petter Selasky wrote:</span><br class=3D"" = style=3D"caret-color: rgb(0, 0, 0); font-family: Menlo-Regular; = font-size: 13px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><blockquote type=3D"cite" class=3D"" style=3D"font-family: = Menlo-Regular; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;">On 4/26/23 12:36, Zhenlei Huang wrote:<br class=3D""><blockquote = type=3D"cite" class=3D"">Hi,<br class=3D""><br class=3D"">I'm recently = working on<span class=3D"Apple-converted-space"> </span><a = href=3D"https://reviews.freebsd.org/D39638" = class=3D"">https://reviews.freebsd.org/D39638</a><span = class=3D"Apple-converted-space"> </span>(sysctl(9): Enable vnet = sysctl variables be loader tunable),<br class=3D"">the changes to = `sys/kern/link_elf_obj.c` are runtime tested, but not those to = `sys/kern/link_elf.c` .<br class=3D""><br class=3D"">After some hacking = I realized that `link_elf.c` is for EXEC (Executable file) or DYN = (Shared object file), and `link_elf_obj.c` is<br class=3D"">for REL = (Relocatable file).<br class=3D""><br class=3D"">```<br class=3D"">/* = link_elf.c */<br class=3D"">static int<br = class=3D"">link_elf_load_file(linker_class_t cls, const char* = filename,<br class=3D""> linker_file_t* = result)<br class=3D"">{<br class=3D"">...<br class=3D""><span = class=3D"Apple-tab-span" style=3D"white-space: pre;"> </span>if = (hdr->e_type !=3D ET_EXEC && hdr->e_type !=3D ET_DYN) {<br = class=3D""><span class=3D"Apple-tab-span" style=3D"white-space: pre;"> = </span><span class=3D"Apple-tab-span" style=3D"white-space: pre;"> = </span>error =3D ENOSYS;<br class=3D""><span class=3D"Apple-tab-span" = style=3D"white-space: pre;"> </span><span class=3D"Apple-tab-span" = style=3D"white-space: pre;"> </span>goto out;<br class=3D""><span = class=3D"Apple-tab-span" style=3D"white-space: pre;"> </span>}<br = class=3D"">...<br class=3D"">}<br class=3D""><br class=3D""><br = class=3D"">/* link_elf_obj.c */<br class=3D"">static int<br = class=3D"">link_elf_load_file(linker_class_t cls, const char = *filename,<br class=3D""> linker_file_t = *result)<br class=3D"">{<br class=3D"">...<br class=3D""><span = class=3D"Apple-tab-span" style=3D"white-space: pre;"> </span>if = (hdr->e_type !=3D ET_REL) {<br class=3D""><span = class=3D"Apple-tab-span" style=3D"white-space: pre;"> </span><span = class=3D"Apple-tab-span" style=3D"white-space: pre;"> </span>error =3D = ENOSYS;<br class=3D""><span class=3D"Apple-tab-span" style=3D"white-space:= pre;"> </span><span class=3D"Apple-tab-span" style=3D"white-space: = pre;"> </span>goto out;<br class=3D""><span class=3D"Apple-tab-span" = style=3D"white-space: pre;"> </span>}<br class=3D"">...<br = class=3D"">}<br class=3D"">```<br class=3D""><br class=3D"">Run the = following snip:<br class=3D"">```<br class=3D""># find /boot/kernel = -type f -name "*.ko" -exec readelf -h {} \; | grep Type<br = class=3D"">```<br class=3D"">shows that all the kernel modules' types = are `REL (Relocatable file)`.<br class=3D""><br class=3D"">I guess if = some module such as if_bridge is linked to DYN type, then I can do = runtime for the changes to `sys/kern/link_elf.c`.<br class=3D""><br = class=3D"">I'm not familiar with elf and linkers, is that ( compile = module and link it to DYN type ) possible ?<br = class=3D""></blockquote></blockquote><br class=3D"" style=3D"caret-color: = rgb(0, 0, 0); font-family: Menlo-Regular; font-size: 13px; font-style: = normal; font-variant-caps: normal; font-weight: 400; letter-spacing: = normal; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span class=3D"" style=3D"caret-color: rgb(0, 0, = 0); font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">Module = file type (shared object vs. object file) depends on = architecture.</span><br class=3D"" style=3D"caret-color: rgb(0, 0, 0); = font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span class=3D"" style=3D"caret-color: rgb(0, 0, = 0); font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">For = amd64 modules are objects, while kernel is shared library.</span><br = class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: = Menlo-Regular; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><span class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: = Menlo-Regular; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none; float: none; display: inline !important;">For arm64 (and all other = arches, I believe) modules and kernels are shared</span><br class=3D"" = style=3D"caret-color: rgb(0, 0, 0); font-family: Menlo-Regular; = font-size: 13px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><span class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: = Menlo-Regular; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none; float: none; display: inline = !important;">libraries.</span></div></blockquote><blockquote type=3D"cite"= class=3D"" style=3D"font-family: Helvetica; font-size: 13px; = font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; orphans: auto; text-align: start; text-indent: = 0px; text-transform: none; white-space: normal; widows: auto; = word-spacing: 0px; -webkit-text-size-adjust: auto; = -webkit-text-stroke-width: 0px; text-decoration: none;"><div = class=3D""><br class=3D"" style=3D"caret-color: rgb(0, 0, 0); = font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span class=3D"" style=3D"caret-color: rgb(0, 0, = 0); font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">I think = you can link amd64 module as shared object, but this require = enough</span><br class=3D"" style=3D"caret-color: rgb(0, 0, 0); = font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span class=3D"" style=3D"caret-color: rgb(0, 0, = 0); font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">hacking = of the build infrastructure. At least I am not aware of a = simple</span><br class=3D"" style=3D"caret-color: rgb(0, 0, 0); = font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span class=3D"" style=3D"caret-color: rgb(0, 0, = 0); font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">knob to = switch the produced type.</span><br class=3D"" style=3D"caret-color: = rgb(0, 0, 0); font-family: Menlo-Regular; font-size: 13px; font-style: = normal; font-variant-caps: normal; font-weight: 400; letter-spacing: = normal; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"></div></blockquote><div style=3D"caret-color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: = normal; font-variant-caps: normal; font-weight: 400; letter-spacing: = normal; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><br class=3D""></div><div = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><div = style=3D"caret-color: rgb(0, 0, 0);" class=3D"">I did some hack on = `sys/conf/kmod.mk` and finally produced DYN kernel modules.</div><div = style=3D"caret-color: rgb(0, 0, 0);" class=3D"">The good news is I do = some basic sysctl tests, but the bad news is the module does not = function correctly.</div><div style=3D"caret-color: rgb(0, 0, 0);" = class=3D""><br class=3D""></div><div style=3D"caret-color: rgb(0, 0, = 0);" class=3D"">For exampe the if_disc.c</div><div style=3D"caret-color: = rgb(0, 0, 0);" class=3D""><br class=3D""></div><div style=3D"caret-color: = rgb(0, 0, 0);" class=3D"">```</div><div class=3D""><div class=3D""><font = class=3D""><span class=3D"" style=3D"caret-color: rgb(0, 0, 0);">static = void</span></font></div><div class=3D""><font class=3D""><span class=3D"" = style=3D"caret-color: rgb(0, 0, 0);">vnet_disc_init(const void *unused = __unused)</span></font></div><div class=3D""><font class=3D""><span = class=3D"" style=3D"caret-color: rgb(0, 0, = 0);">{</span></font></div><div class=3D""><span class=3D"Apple-tab-span" = style=3D"caret-color: rgb(0, 0, 0); white-space: pre;"> </span>/* = Reference V_disc_cloner will immediately trigger page fault panic = */</div><div class=3D""><font class=3D""><span class=3D"" = style=3D"caret-color: rgb(0, 0, 0);"><span class=3D"Apple-tab-span" = style=3D"white-space: pre;"> </span>V_disc_cloner =3D = if_clone_simple(discname, disc_clone_create,</span></font></div><div = class=3D""><font class=3D""><span class=3D"" style=3D"caret-color: = rgb(0, 0, 0);"><span class=3D"Apple-tab-span" style=3D"white-space: = pre;"> </span><span class=3D"Apple-converted-space"> </span> = disc_clone_destroy, 0);</span></font></div><div class=3D""><font = class=3D""><span class=3D"" style=3D"caret-color: rgb(0, 0, = 0);">}</span></font></div><div class=3D""><font class=3D""><span = class=3D"" style=3D"caret-color: rgb(0, 0, = 0);">VNET_SYSINIT(vnet_disc_init, SI_SUB_PSEUDO, = SI_ORDER_ANY,</span></font></div><div class=3D""><font class=3D""><span = class=3D"" style=3D"caret-color: rgb(0, 0, 0);"> = vnet_disc_init, NULL);</span></font></div></div><div style=3D"caret-color:= rgb(0, 0, 0);" class=3D"">```</div><div style=3D"caret-color: rgb(0, 0, = 0);" class=3D""><br class=3D""></div></div><div style=3D"caret-color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: = normal; font-variant-caps: normal; font-weight: 400; letter-spacing: = normal; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D"">I suspect the relocation is not done = correctly for <span class=3D"" style=3D"caret-color: rgb(0, 0, = 0);">DYN elf kmod on amd64.</span></div><div style=3D"caret-color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: = normal; font-variant-caps: normal; font-weight: 400; letter-spacing: = normal; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><span class=3D"" style=3D"caret-color: = rgb(0, 0, 0);"><br class=3D""></span></div><div style=3D"caret-color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 13px; font-style: = normal; font-variant-caps: normal; font-weight: 400; letter-spacing: = normal; text-align: start; text-indent: 0px; text-transform: none; = white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><span class=3D"" style=3D"caret-color: = rgb(0, 0, 0);">My local patch to kmod.mk:</span></div><div = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><span = class=3D"" style=3D"caret-color: rgb(0, 0, 0);"><br = class=3D""></span></div><div style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><span class=3D"" style=3D"caret-color: = rgb(0, 0, 0);">```</span></div><div style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><div class=3D""><font class=3D""><span = class=3D"" style=3D"caret-color: rgb(0, 0, 0);">diff --git = a/sys/conf/kmod.mk b/sys/conf/kmod.mk</span></font></div><div = class=3D""><font class=3D""><span class=3D"" style=3D"caret-color: = rgb(0, 0, 0);">index 134b150af1d9..1fc5386204a5 = 100644</span></font></div><div class=3D""><font class=3D""><span = class=3D"" style=3D"caret-color: rgb(0, 0, 0);">--- = a/sys/conf/kmod.mk</span></font></div><div class=3D""><font = class=3D""><span class=3D"" style=3D"caret-color: rgb(0, 0, 0);">+++ = b/sys/conf/kmod.mk</span></font></div><div class=3D""><font = class=3D""><span class=3D"" style=3D"caret-color: rgb(0, 0, 0);">@@ = -84,6 +84,7 @@ __KLD_SHARED=3Dyes</span></font></div><div class=3D""><font= class=3D""><span class=3D"" style=3D"caret-color: rgb(0, 0, = 0);"> .else</span></font></div><div class=3D""><font class=3D""><span= class=3D"" style=3D"caret-color: rgb(0, 0, = 0);"> __KLD_SHARED=3Dno</span></font></div><div class=3D""><font = class=3D""><span class=3D"" style=3D"caret-color: rgb(0, 0, = 0);"> .endif</span></font></div><div class=3D""><font = class=3D""><span class=3D"" style=3D"caret-color: rgb(0, 0, = 0);">+__KLD_SHARED=3Dyes</span></font></div><div class=3D""><font = class=3D""><span class=3D"" style=3D"caret-color: rgb(0, 0, = 0);"> </span></font></div><div class=3D""><font class=3D""><span = class=3D"" style=3D"caret-color: rgb(0, 0, 0);"> .if = !empty(CFLAGS:M-O[23s]) && = empty(CFLAGS:M-fno-strict-aliasing)</span></font></div><div = class=3D""><font class=3D""><span class=3D"" style=3D"caret-color: = rgb(0, 0, 0);"> CFLAGS+=3D = -fno-strict-aliasing</span></font></div><div class=3D""><font = class=3D""><span class=3D"" style=3D"caret-color: rgb(0, 0, 0);">@@ = -167,6 +168,7 @@ CFLAGS+=3D -fno-omit-frame-pointer = -mno-omit-leaf-frame-pointer</span></font></div><div class=3D""><font = class=3D""><span class=3D"" style=3D"caret-color: rgb(0, 0, 0);"> = ${MACHINE_CPUARCH} =3D=3D "powerpc"</span></font></div><div = class=3D""><font class=3D""><span class=3D"" style=3D"caret-color: = rgb(0, 0, 0);"> CFLAGS+=3D = -fPIC</span></font></div><div class=3D""><font class=3D""><span class=3D""= style=3D"caret-color: rgb(0, 0, = 0);"> .endif</span></font></div><div class=3D""><font = class=3D""><span class=3D"" style=3D"caret-color: rgb(0, 0, = 0);">+CFLAGS+=3D -fPIC</span></font></div><div = class=3D""><font class=3D""><span class=3D"" style=3D"caret-color: = rgb(0, 0, 0);"> </span></font></div><div class=3D""><font = class=3D""><span class=3D"" style=3D"caret-color: rgb(0, 0, 0);"> # = Temporary workaround for PR 196407, which contains the fascinating = details.</span></font></div><div class=3D""><font class=3D""><span = class=3D"" style=3D"caret-color: rgb(0, 0, 0);"> # Don't allow = clang to use fpu instructions or registers in kernel = modules.</span></font></div></div><div style=3D"caret-color: rgb(0, 0, = 0); font-family: Helvetica; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><span class=3D"" style=3D"caret-color: = rgb(0, 0, 0);">```</span></div><div style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><br class=3D""></div><div = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><br = class=3D""></div><div style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;" class=3D"">As for <a = href=3D"https://reviews.freebsd.org/D39638" = class=3D"">https://reviews.freebsd.org/D39638</a>, for other platform = such as arm, I think the `link_elf_propagate_vnets()` should work if = `parse_vnet()` works.</div><div style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;" class=3D""><br class=3D""></div><div = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13px; font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D"">I'll = appreciate if someone can test it on platforms those have DYN type = kernel modules. </div></div></blockquote><div><br = class=3D""></div><div>Good news on this!</div><div><br = class=3D""></div><div>I've managed to test the change for `link_elf.c` = on QEMU RISCV . It works as expected and solid !</div><div><br = class=3D""></div><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D""><br class=3D"" style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><blockquote type=3D"cite" class=3D"" = style=3D"font-family: Helvetica; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; = text-decoration: none;"><div class=3D""><br class=3D"" = style=3D"caret-color: rgb(0, 0, 0); font-family: Menlo-Regular; = font-size: 13px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><br class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: = Menlo-Regular; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><blockquote type=3D"cite" class=3D"" style=3D"font-family: = Menlo-Regular; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><blockquote type=3D"cite" class=3D""><br = class=3D""></blockquote><br class=3D"">Hi,<br class=3D""><br class=3D"">I = don't have an answer for you either, but I have seen in the past, = loading<br class=3D"">kernel modules behaves a bit like libraries, in = the following regard:<br class=3D""><br class=3D"">If two kernel modules = define the same global symbol, then no warning is<br class=3D"">given = and the first loaded symbol definition (I think) is used to resolve<br = class=3D"">that symbol for all kernel modules, regardless of the = prototype. Probably we<br class=3D"">should not allow this. That's why = building LINT is a good thing, to avoid<br class=3D"">this issue.<br = class=3D""></blockquote><span class=3D"" style=3D"caret-color: rgb(0, 0, = 0); font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">No, = in-kernel linker does not behave this way.</span><br class=3D"" = style=3D"caret-color: rgb(0, 0, 0); font-family: Menlo-Regular; = font-size: 13px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><span class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: = Menlo-Regular; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none; float: none; display: inline !important;">Modules need to contain = explicit reference to all modules they depend upon,</span><br class=3D"" = style=3D"caret-color: rgb(0, 0, 0); font-family: Menlo-Regular; = font-size: 13px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><span class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: = Menlo-Regular; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none; float: none; display: inline !important;">using the = MODULE_DEPEND() macro. Only symbols from the dependencies = are</span><br class=3D"" style=3D"caret-color: rgb(0, 0, 0); = font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span class=3D"" style=3D"caret-color: rgb(0, 0, = 0); font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline = !important;">resolved.</span><br class=3D"" style=3D"caret-color: rgb(0, = 0, 0); font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><br class=3D"" style=3D"caret-color: rgb(0, 0, = 0); font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span class=3D"" style=3D"caret-color: rgb(0, 0, = 0); font-family: Menlo-Regular; font-size: 13px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">All = modules get an implicit reference to kernel.</span><br class=3D"" = style=3D"caret-color: rgb(0, 0, 0); font-family: Menlo-Regular; = font-size: 13px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><br class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: = Menlo-Regular; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><blockquote type=3D"cite" class=3D"" style=3D"font-family: = Menlo-Regular; font-size: 13px; font-style: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><br class=3D"">Even if we don't have C++ support in the FreeBSD = kernel, defining symbol<br class=3D"">names the way C++ does for C could = be nice for the kernel too, also with<br class=3D"">regards to debugging = systems.<br class=3D""><br class=3D"">Many times when I don't know what = is going on, I do like this:<br class=3D""><br class=3D"">#include = <sys/kdb.h><br class=3D""><br class=3D"">....<br class=3D""><br = class=3D"">if (not too fast or my sysctl debug) {<br = class=3D""> printf("My tracer\n");<br = class=3D""> kdb_backtrace();<br class=3D"">}<br class=3D""><br = class=3D"">Dtrace can also do this, but not during boot. Just track who = is calling<br class=3D"">those functions, and you'll probably find the = answer to your question!<br class=3D""><br class=3D"">--HPS<br = class=3D""><br class=3D""><blockquote type=3D"cite" class=3D""><br = class=3D"">Best regards,<br = class=3D"">Zhenlei</blockquote></blockquote></div></blockquote></div></blo= ckquote></div><div class=3D""><div><br = class=3D""></div></div></body></html>= --Apple-Mail=_41DA2938-02A5-4B92-9856-2D3CD7B51E9C--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B2F70E53-BC01-4E13-925F-589CF1A9B5B7>