Date: Sat, 19 Jul 2008 16:12:36 +0200 From: Lars Engels <lme@FreeBSD.org> To: Jeff Roberson <jroberson@jroberson.net> Cc: arch@freebsd.org Subject: Re: witness performance improvements Message-ID: <20080719141236.GF56464@e.0x20.net> In-Reply-To: <20080718163231.B954@desktop> References: <20080718163231.B954@desktop>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] On Fri, Jul 18, 2008 at 04:41:58PM -1000, Jeff Roberson wrote: > Hello, > > I have a patch that improves witness performance available at: > > http://people.freebsd.org/~jeff/witness.diff > > This improvement comes at the cost of some significant space overhead. > It changes the witness graph from a linked tree to a matrix based > approach. Relationships can be quickly resolved with a table lookup. > The table size is WITNESS_COUNT^2, or 1MB with the current count of > 1024. > > This patch also makes struct witness objects persistent even after the > last lock using this name has been removed. This is helpful for short > lived objects which may be created frequently. > > To reduce lock contention on SMP witness_checkorder() now runs without > the w_mtx when there are no lock violations. I also cache a > lock_list_entry in each thread as allocating these requires the w_mtx. > The entry is disposed of at thread_exit(). > > There is also a new sysctl that produces dot output which graphs lock > order relationships with the graphviz program. > > Most of this work was done by Ilya Maykov while he was at Isilon > systems. The locking work and some cleanup/porting/refinement was done > by me on behalf of Nokia. > > The performance improvement can be significant. It is only on the > order of 10-20% for buildkernel but on a packet forwarding test at > nokia it sped things up by 5x putting a witness enabled kernel within > about 50% of the performance of a kernel without. I believe > buildworld isn't helped as much because forking and exiting a lot > would then contend on the witness lock. > > I'm mostly interested in hearing what people have to say about the > space bloat. I believe it is in a commit ready state. > > Thanks, > Jeff The speed improvement is significant here (Core Duo Machine). The kernel build time went from ~8:30 min to 5:30 min. But when I run sysctl -a the kernel panics. kgdb output: Unread portion of the kernel message buffer: panic: Assertion mcount == fcount failed at /usr/src/sys/kern/subr_witness.c:2882 cpuid = 1 KDB: enter: panic panic: from debugger cpuid = 1 KDB: stack backtrace: Physical memory: 1002 MB Dumping 68 MB: 53 37 21 5 (kgdb) bt #0 doadump () at pcpu.h:196 #1 0xc0476c69 in db_fncall (dummy1=-1061959744, dummy2=0, dummy3=3, dummy4=0xe54ce8ec "") at /usr/src/sys/ddb/db_command.c:516 #2 0xc0477218 in db_command (last_cmdp=0xc099a9b0, cmd_table=0x0, dopager=1) at /usr/src/sys/ddb/db_command.c:413 #3 0xc047734a in db_command_loop () at /usr/src/sys/ddb/db_command.c:466 #4 0xc0478b3d in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_main.c:228 #5 0xc0618d56 in kdb_trap (type=3, code=0, tf=0xe54cea94) at /usr/src/sys/kern/subr_kdb.c:534 #6 0xc087ec36 in trap (frame=0xe54cea94) at /usr/src/sys/i386/i386/trap.c:683 #7 0xc08634bb in calltrap () at /usr/src/sys/i386/i386/exception.s:165 #8 0xc0618eda in kdb_enter (why=0xc08dc49c "panic", msg=0xc08dc49c "panic") at cpufunc.h:60 #9 0xc05ebe9c in panic (fmt=0xc08d72ed "Assertion %s failed at %s:%d") at /usr/src/sys/kern/kern_shutdown.c:556 #10 0xc062bf62 in sysctl_debug_witness_cyclegraph (oidp=0xc096bec0, arg1=0x0, arg2=0, req=0xe54ceba4) at /usr/src/sys/kern/subr_witness.c:2882 #11 0xc05f59d7 in sysctl_root (oidp=Variable "oidp" is not available. ) at /usr/src/sys/kern/kern_sysctl.c:1325 #12 0xc05f5b05 in userland_sysctl (td=0xc49c08c0, name=0xe54cec10, namelen=3, old=0x0, oldlenp=0xbfbfd9b0, inkernel=0, new=0x0, newlen=0, retval=0xe54cec70, flags=0) at /usr/src/sys/kern/kern_sysctl.c:1420 #13 0xc05f5f4c in __sysctl (td=0xc49c08c0, uap=0xe54cecf8) at /usr/src/sys/kern/kern_sysctl.c:1355 #14 0xc087e3b3 in syscall (frame=0xe54ced38) at /usr/src/sys/i386/i386/trap.c:1081 #15 0xc0863520 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:261 #16 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) The kernel sources are one week old. [-- Attachment #2 --] -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkiB9lQACgkQKc512sD3afh1XgCfUuP0qvBLuE+GYN1lF3iEdkxU wmYAoIP2aIDjccTcVViLbY09WUWZsiaL =kD/c -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080719141236.GF56464>