Date: Tue, 29 Jul 2014 19:35:46 +0000 From: bugzilla-noreply@freebsd.org To: freebsd-bugs@FreeBSD.org Subject: [Bug 192249] New: [PATCH] Load reloc modules (amd64, mips) faster by caching global syms Message-ID: <bug-192249-8@https.bugs.freebsd.org/bugzilla/>
next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192249 Bug ID: 192249 Summary: [PATCH] Load reloc modules (amd64, mips) faster by caching global syms Product: Base System Version: 11.0-CURRENT Hardware: Any OS: Any Status: Needs Triage Severity: Affects Only Me Priority: --- Component: kern Assignee: freebsd-bugs@FreeBSD.org Reporter: conrad.meyer@isilon.com Created attachment 145126 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=145126&action=edit (cd src/ && patch -p1 ; diff is against recent-ish CURRENT) The problem: At Isilon, efs.ko is a very large kernel module; we were seeing kldload take 12-13 seconds. Most of that time was spent repeatedly resolving the same global symbols for relocations. link_elf_obj would lookup the same globals many times; each time, it would lookup in the local module first, before missing and falling back to deps, where it would find 'kernel' and get a fast hit (hash table hit). Since ld(1) doesn't generate lookup hashes (SysV or GNU) for relocatable objects (kmods on AMD64), lookups in the local module involved walking the entire symtab and strcmp()ing. efs.ko has 32k symbols. This was very slow. Solution: Add a special ELF SHN_ value (in the SHN_LOOS to SHN_HIOS range) to represent cached SHN_UNDEF globals. Cache the results of global lookup during relocatable object load. Modify Dtrace to ignore these SHN_FBSD_CACHED symbols the same way it ignored SHN_UNDEF symbols before. I didn't find other users of SHN_UNDEF that might be negatively affected by this change. Testing done: Basic Dtrace testing suggests it isn't broken. Performance results: Before # for i in 1 2 3 ; do time kldload efs.ko ; kldunload efs.ko ; done kldload efs.ko 0.00s user 13.00s system 103% cpu 12.607 total kldload efs.ko 0.00s user 13.25s system 102% cpu 12.876 total kldload efs.ko 0.00s user 13.14s system 103% cpu 12.752 total kldload zfs.ko 0.00s user 1.77s system 102% cpu 1.729 total # Smaller, for comparison After kldload efs.ko 0.00s user 0.25s system 99% cpu 0.252 total kldload efs.ko 0.00s user 0.20s system 99% cpu 0.204 total kldload efs.ko 0.00s user 0.21s system 99% cpu 0.211 total kldload zfs.ko 0.00s user 0.04s system 96% cpu 0.040 total # For comparison Sponsored by: EMC / Isilon storage division -- You are receiving this mail because: You are the assignee for the bug.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-192249-8>