Date: Mon, 29 Sep 2008 09:59:36 -0700 From: Dilip Chhetri <dchhetri@panasas.com> To: Tijl Coosemans <tijl@ulyssis.org> Cc: freebsd-threads@freebsd.org Subject: Re: getting stack trace for other thread on the same process : libthr Message-ID: <48E10978.2090907@panasas.com> In-Reply-To: <200809262331.29353.tijl@ulyssis.org> References: <48DD32D2.2060304@panasas.com> <200809262331.29353.tijl@ulyssis.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Tijl Coosemans wrote: > On Friday 26 September 2008 21:06:58 Dilip Chhetri wrote: > >>Question >>-------- >> My program is linked with libthr in FreeBSD-7.0. The program has >>in the order of 20 threads, and a designated monitoring thread at >>some point wants to know what are other/stuck threads doing. This >>needs to be done by printing stack backtrace for the thread to >>stdout. >> >> I understand pthread_t structure has pointer to the target >>thread's stack, but to get the trace I need to know value of >>stack-pointer register and base-pointer register. I looked at the >>code and I don't find any mechanism by which I could read the target >>threads register context (because it all resides within kernel thread >>structure). Further code study reveals that kernel_thread->td_frame >>contains the register context for a thread, but is valid only when >>the thread is executing/sleeping inside the kernel. >> >> Is there anything I'm missing here ? Is there an easy way to >>traverse stack for some thread with in the same process. >> >> I considered/considering following approaches, >>a) use PTRACE >> ruled out, because you can't trace the process from within the >> same process >> >>b) somehow temporarily stop the target-thread and read td_frame by >> traversing kernel data structure through /dev/kmem. After doing >> stack traversal resume the target thread. >> >> >>Detailed problem background >>-------------------------- >> We have this process X with ~20 threads, each processing some >>requests. One of them is designated as monitoring/dispatcher thread. >>When a new request arrives, dispatcher thread tries to queue the task >>to idle thread. But if all threads are busy processing requests, the >>dispatcher thread is supposed to print the stack back trace for each >>of the busy thread. This is our *debugging* mechanism to find >>potential fault-points. >> >> In FreeBSD-4.6.2, we hacked libc_r:pthread_t to achieve our goal. >>But in FreeBSD-7.0, we decided to use libthr and hack doesn't seem to >>be easy. >> >>Target setup >>------------ >> * SMP : around 8 CPU >> * process : it's going to be run as root and have around ~20 threads > > > You could try registering a signal handler for SIGUSR1 that prints a > stack backtrace using the stack pointer in the sigcontext and then call > pthread_kill(SIGUSR1) on whichever thread you want a backtrace of. Thanks, but as I mentioned it's a network based program and it may be sleeping/stuck in syscall for some packets, in this case pthread_kill will not work because signals are delivered only when you return from syscall (that's what I haved learned from old UNIX books in my college).
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48E10978.2090907>