Date: Fri, 1 Apr 2016 00:23:37 +0300 From: Aleksander Alekseev <mail@eax.me> To: Mark Johnston <markj@FreeBSD.org> Cc: Adrian Chadd <adrian.chadd@gmail.com>, Hans Petter Selasky <hps@selasky.org>, Andriy Voskoboinyk <s3erios@gmail.com>, freebsd-dtrace@freebsd.org, "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Re: I need a little help in fixing `exclusive sleep mutex urtwn0_com_lock` in CURRENT Message-ID: <20160401002337.36700d9b@portege> In-Reply-To: <20160331180333.GA25235@wkstn-mjohnston.west.isilon.com> References: <20160330123048.3361a9e4@fujitsu> <op.ye4x9fosiew4ia@localhost> <56FBBC62.6040905@selasky.org> <CAJ-Vmon=s-uKMhzgRm--DttXJ8_x_cAKnpALu4UeLOdQQ9nXpw@mail.gmail.com> <20160331204256.5cb1fdaf@portege> <20160331180333.GA25235@wkstn-mjohnston.west.isilon.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> > By any chance is it possible to trace mtx_lock / mtx_unlock calls > > using DTrace? I see number of probes in `dtrace -l` which look like > > something I need. Unfortunately they are named like knlist_mtx_lock > > or do_lock_umutex so I can't figure out whether these are probes I > > looking for or not. > > Yes. You can use the lockstat provider to trace lock events: > "dtrace -l -P lockstat". Predicates can be used to isolate events > related to a specific lock. For example: > > # dtrace -n > 'lockstat:::adaptive-acquire /args[0]->lock_object.lo_name == > "so_snd"/{stack();}' > > will print a stack every time a socket send buffer lock is acquired. > In general, the lock name is specified as an initialization parameter. > In your case it'll be "urtwn0_com_lock". > > lockstat(1) is a command-line program that's good at aggregating data > collected from lockstat probes; for the type of debugging you're doing > it's probably not very useful. > > Note that DTrace probably isn't very helpful here if the panic occurs > immediately after the event you're interested in occurs, since > dtrace(1) won't have time to retrieve the trace record and print it. Thanks, Mark. Here is a funny thing. I thought that by writing debug.witness.watch=0 to /etc/sysctl.conf and rebooting I will give DTrace some time to report who acquired a lock and didn't released it. But in this case kernel crashed differently (I checked twice): http://pastebin.com/raw/f08Fd70B And here is saved DTrace log for lock "urtwn0_com_lock": http://pastebin.com/raw/0X9zgeiY If I'm not wrong everything is OK. No missing mtx_unlock this time. Not sure what to do next so far. And it's pretty late here in UTC+3 already so I will think about this tomorrow. Any advices would be appreciated. -- Best regards, Aleksander Alekseev http://eax.me/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20160401002337.36700d9b>