Date: Wed, 11 Feb 2004 12:39:50 -0800 From: Peter Wemm <peter@wemm.org> To: Petri Helenius <pete@he.iki.fi>, obrien@freebsd.org Cc: freebsd-amd64@freebsd.org Subject: Re: kld support Message-ID: <200402111239.50639.peter@wemm.org> In-Reply-To: <4029EF67.9020703@he.iki.fi> References: <4029EF67.9020703@he.iki.fi>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wednesday 11 February 2004 01:01 am, Petri Helenius wrote: > Since the kld support for amd64 is pending, is there a way to compile > netgraph modules statically into the kernel to get for example > ng_ether support? > > I would also be happy to try out early code if such thing exists. (to > enable KLD) For the time being, I have a spare opteron box. I have bad news and some good news on this front. I thought we might be able to get away with a quick hack to binutils to get it to work. Indeed, I've been able to produce .ko files that can be loaded and almost even work. But thats the problem... almost. Check out these lines in sys/conf/NOTES: options NETGRAPH #netgraph(4) system options NETGRAPH_ASYNC options NETGRAPH_BPF options NETGRAPH_BRIDGE options NETGRAPH_CISCO options NETGRAPH_ECHO options NETGRAPH_ETHER options NETGRAPH_FRAME_RELAY options NETGRAPH_GIF .....[lots more].... Thats how you chose what you want to compile in. Anyway, the problem with binutils is subtle but massive. There is a sanity check that prevents you producing shared libs without -fpic mode. Because *everybody* knows that ld.so cannot guarantee that it can load the library within the first 2GB of address space on linux, so binutils enforces this policy. The problem though is that all the wold is not linux. And there are legitimate reasons for doing this. One of which is our kernel modules where we can provide those guarantees. Anyway, removing the sanity check lets you produce them. BUT... because those code paths have never been tested on linux, binutils neglects to support the rest of the relocation types that would be needed, and none of the linux folks have noticed. Binutils is a freaking nightmare to try and work in, and it has me totally beaten on this one. So, I thought.. since I can do something about the in-kernel linker, perhaps I can add PIC mode support? The problem with that though is that gcc says "-fPIC mode not implemented in kernel mode" and errors out. There's no way in hell that I'm going to try and implement that in gcc either, so plan B is a bust. The good news though is that Plan C is possible. Plan C is to stop using -shared for kernel modules and use a simple .o file, like linux does. I think I can write code for the kld subsystem without too much pain. And because linux does the same thing already, we dont have to worry about treading on landmines in binutils/gcc, because that path should be well trodden already. We already produce these files in kmod.mk. If you have a look at the obj dirs, we have: ld -d -warn-common -r -d -o if_sl.kld if_sl.o slcompress.o touch /home/src/sys/modules/if_sl/obj/export_syms awk -f /home/src/sys/modules/if_sl/../../conf/kmod_syms.awk if_sl.kld /home/src/sys/modules/if_sl/obj/export_syms | xargs -J% objcopy % if_sl.kld ld -Bshareable -d -warn-common -o if_sl.ko if_sl.kld ld: if_sl.kld: relocation R_X86_64_32S can not be used when making a shared object; recompile with -fPIC .... ls obj/ -rw-r--r-- 1 root src 19608 Feb 11 12:32 if_sl.kld That file was the immediate step prior to conversion to a .ko file, and thats what we can load. On another machine with the hacks to produce a .ko anyway, we have: peter@hammer[12:37pm]/home/obj/home/src/sys/modules/if_sl-105> ls -l if_sl.* -rw-r--r-- 1 root wheel 19608 Feb 6 14:46 if_sl.kld -rwxr-xr-x 1 root wheel 23179 Feb 6 14:46 if_sl.ko* -rw-r--r-- 1 root wheel 15760 Feb 6 14:46 if_sl.o -rw-r--r-- 1 root wheel 24668 Feb 6 14:46 if_sl.s Also of note is that the .kld file is smaller than the .ko file. So yes, there is light at the end of the tunnel. I think its time to give up on binutils/gcc hacking and take the path of least resistence and write some code that we control. The original reasons for having -shared kernel object files are mostly obsolete these days. We dont use the DT_NEEDED tags for dependency tracking anymore, for example. I think we can actually get a superior system as a result. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200402111239.50639.peter>