Date: Fri, 8 Jun 2007 10:56:30 -0700 From: Peter Wemm <peter@wemm.org> To: freebsd-amd64@freebsd.org Subject: Re: amd64 kmod symbol relocation Message-ID: <200706081056.30475.peter@wemm.org> In-Reply-To: <46674245.9020304@netapp.com> References: <46674245.9020304@netapp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wednesday 06 June 2007 16:24, Dixit, Amol wrote: > Hi, > I am having issues loading a test module on amd64 platform, ONLY when > its loaded below 2gb (0x80000000). > The module is trying to access a function 'printf()' in a shared > library which is beyond the 2gb boundary but the runtime loader is > trying to relocate the symbol to some other offset in its .text > section. > > (gdb) disass alloc_loader > Dump of assembler code for function alloc_loader: > 0xffffff000b3a2000 <alloc_loader+0>: push %rbp > 0xffffff000b3a2001 <alloc_loader+1>: mov $0xb3a2026,%rdi > 0xffffff000b3a2008 <alloc_loader+8>: xor %eax,%eax > 0xffffff000b3a200a <alloc_loader+10>: mov %rsp,%rbp > 0xffffff000b3a200d <alloc_loader+13>: callq 0x8024b730 > <---- call printf() > 0xffffff000b3a2012 <alloc_loader+18>: leaveq > 0xffffff000b3a2013 <alloc_loader+19>: xor %eax,%eax > 0xffffff000b3a2015 <alloc_loader+21>: retq > > (gdb) p/x frame.tf_rip > $10 = 0xffffff008024b730 <--- panic "privileged > instruction fault" address > (gdb) info sym 0xffffff008024b730 > No symbol matches 0xffffff008024b730. <--- nothing here, hence > the panic! > (gdb) info sym 0xffffffff8024b730 > printf in section .text <--- here it is! > beyond 2gb (sign-extended) > > The module is compiled with kernel memory model (mcmodel=kernel). Any > ideas why the relocation is failing in this manner? How do we force > kldload to use addresses above 2gb? > > Object code looks like this: > > alloc_loader(): > 0: 55 push %rbp > 1: 48 c7 c7 00 00 00 00 mov $0x0,%rdi > 4: R_X86_64_32S .rodata.str1.1+0x6 > 8: 31 c0 xor %eax,%eax > a: 48 89 e5 mov %rsp,%rbp > d: e8 00 00 00 00 callq 12 <alloc_loader+0x12> > e: R_X86_64_PC32 > printf+0xfffffffffffffffc 12: c9 leaveq > 13: 31 c0 xor %eax,%eax > 15: c3 retq > > I believe relocation types R_X86_64_32S and R_X86_64_PC32 require top > 33 bits 1 or 0 and in case of 'printf' 0xffffff008024b730 doesn't > obey it...I guess! > > Thanks, > Amol OK, you're running into the same problems we had at work. It sounds like you're doing some evil things that are not in standard freebsd. I'm guessing that you've extended kvm considerably so that it is now no longer restricted to 2GB. The problem is that -mcmodel=kernel generates code that only allows for immediate asm offsets of +/- 31 bits. Consider the instruction in the trace above for "callq alloc_loader". Note that it is e8 00 00 00 00 - 4 bytes only! Not 64 bit. If you are going to have modules loaded outside of that 2gb of space, then you will need to compile with -fpic. You need a tiny tweak to the loader to allow this, but it works. -fpic causes the compiler to make all external function calls and data references via indirect lookups, and then relocations are generated against a 64 bit jump (PLT) and offset table (GOT). That allows the kld to make immediate mode jumps and data references to something that is more than 2GB away. There is a small performance penalty for using -fpic mode (but NOT as much of a penalty as it is in i386 mode!), but it still isn't free. Your only alternative is to come up with a way of giving the loader a way to allocate space in that top 2gb of kvm. We use -fpic kld modules at work FWIW. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200706081056.30475.peter>