From owner-freebsd-stable@FreeBSD.ORG Thu Jul 7 12:03:56 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 588641065675; Thu, 7 Jul 2011 12:03:56 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id CC07A8FC1B; Thu, 7 Jul 2011 12:03:53 +0000 (UTC) Received: from [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a] (saphire3.sentex.ca [IPv6:2607:f3e0:0:4:f025:8813:7603:7e4a]) by smarthost1.sentex.ca (8.14.4/8.14.4) with ESMTP id p67C3pf1061169; Thu, 7 Jul 2011 08:03:51 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <4E15A08C.6090407@sentex.net> Date: Thu, 07 Jul 2011 08:03:24 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Kostik Belousov References: <4E154A63.90600@sentex.net> <4E15620A.9030608@FreeBSD.org> <20110707082027.GX48734@deviant.kiev.zoral.com.ua> <4E159959.2070401@sentex.net> In-Reply-To: <4E159959.2070401@sentex.net> X-Enigmail-Version: 1.1.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.67 on IPv6:2607:f3e0:0:1::12 Cc: FreeBSD-STABLE Mailing List , Andriy Gapon Subject: Re: panic: spin lock held too long (RELENG_8 from today) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jul 2011 12:03:56 -0000 On 7/7/2011 7:32 AM, Mike Tancsa wrote: > On 7/7/2011 4:20 AM, Kostik Belousov wrote: >> >> BTW, we had a similar panic, "spinlock held too long", the spinlock >> is the sched lock N, on busy 8-core box recently upgraded to the >> stable/8. Unfortunately, machine hung dumping core, so the stack trace >> for the owner thread was not available. >> >> I was unable to make any conclusion from the data that was present. >> If the situation is reproducable, you coulld try to revert r221937. This >> is pure speculation, though. > > Another crash just now after 5hrs uptime. I will try and revert r221937 > unless there is any extra debugging you want me to add to the kernel > instead ? > > This is an inbound mail server so a little disruption is possible > > kgdb /usr/obj/usr/src/sys/recycle/kernel.debug vmcore.13 > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "i386-marcel-freebsd"... > > Unread portion of the kernel message buffer: > spin lock 0xc0b1d200 (sched lock 1) held by 0xc5dac2e0 (tid 100109) too long > panic: spin lock held too long > cpuid = 0 > Uptime: 5h37m43s > Physical memory: 2035 MB > Dumping 260 MB: 245 229 213 197 181 165 149 133 117 101 85 69 53 37 21 5 And the second crash from today kgdb /usr/obj/usr/src/sys/recycle/kernel.debug vmcore.13 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"... Unread portion of the kernel message buffer: spin lock 0xc0b1d200 (sched lock 1) held by 0xc5dac2e0 (tid 100109) too long panic: spin lock held too long cpuid = 0 Uptime: 5h37m43s Physical memory: 2035 MB Dumping 260 MB: 245 229 213 197 181 165 149 133 117 101 85 69 53 37 21 5 Reading symbols from /boot/kernel/amdsbwd.ko...Reading symbols from /boot/kernel/amdsbwd.ko.symbols...done. done. Loaded symbols for /boot/kernel/amdsbwd.ko #0 doadump () at pcpu.h:231 231 pcpu.h: No such file or directory. in pcpu.h (kgdb) tid 100109 [Switching to thread 82 (Thread 100109)]#0 sched_switch (td=0xc5dac2e0, newtd=0xc553c5c0, flags=260) at /usr/src/sys/kern/sched_ule.c:1866 1866 cpuid = PCPU_GET(cpuid); (kgdb) list 1861 /* 1862 * We may return from cpu_switch on a different cpu. However, 1863 * we always return with td_lock pointing to the current cpu's 1864 * run queue lock. 1865 */ 1866 cpuid = PCPU_GET(cpuid); 1867 tdq = TDQ_CPU(cpuid); 1868 lock_profile_obtain_lock_success( 1869 &TDQ_LOCKPTR(tdq)->lock_object, 0, 0, __FILE__, __LINE__); 1870 #ifdef HWPMC_HOOKS (kgdb) p *td $1 = {td_lock = 0xc0b1d200, td_proc = 0xc5db4000, td_plist = {tqe_next = 0xc5dac5c0, tqe_prev = 0xc5dac008}, td_runq = {tqe_next = 0x0, tqe_prev = 0xc0b1d334}, td_slpq = {tqe_next = 0x0, tqe_prev = 0xc65d3b00}, td_lockq = {tqe_next = 0x0, tqe_prev = 0xc51f6b38}, td_cpuset = 0xc5533e38, td_sel = 0x0, td_sleepqueue = 0xc65d3b00, td_turnstile = 0xc63ceb80, td_umtxq = 0xc5d229c0, td_tid = 100109, td_sigqueue = { sq_signals = {__bits = {0, 0, 0, 0}}, sq_kill = {__bits = {0, 0, 0, 0}}, sq_list = {tqh_first = 0x0, tqh_last = 0xc5dac340}, sq_proc = 0xc5db4000, sq_flags = 1}, td_flags = 4, td_inhibitors = 0, td_pflags = 2097152, td_dupfd = 0, td_sqqueue = 0, td_wchan = 0x0, td_wmesg = 0x0, td_lastcpu = 0 '\0', td_oncpu = 1 '\001', td_owepreempt = 0 '\0', td_tsqueue = 0 '\0', td_locks = -291, td_rw_rlocks = 0, td_lk_slocks = 0, td_blocked = 0x0, td_lockname = 0x0, td_contested = {lh_first = 0x0}, td_sleeplocks = 0x0, td_intr_nesting_level = 0, td_pinned = 0, td_ucred = 0xc5538100, td_estcpu = 0, td_slptick = 0, td_blktick = 0, td_ru = {ru_utime = {tv_sec = 0, tv_usec = 0}, ru_stime = {tv_sec = 0, tv_usec = 0}, ru_maxrss = 1048, ru_ixrss = 85216, ru_idrss = 3834720, ru_isrss = 681728, ru_minflt = 0, ru_majflt = 0, ru_nswap = 0, ru_inblock = 82, ru_oublock = 271222, ru_msgsnd = 135625, ru_msgrcv = 2427350, ru_nsignals = 0, ru_nvcsw = 2076938, ru_nivcsw = 731134}, td_incruntime = 852332612, td_runtime = 88202475877, td_pticks = 5326, td_sticks = 48, td_iticks = 0, td_uticks = 0, td_intrval = 0, td_oldsigmask = {__bits = {0, 0, 0, 0}}, td_sigmask = {__bits = {0, 0, 0, 0}}, td_generation = 2808072, td_sigstk = {ss_sp = 0x0, ss_size = 0, ss_flags = 0}, td_xsig = 0, td_profil_addr = 0, td_profil_ticks = 0, td_name = "nfsd: service\000\000\000\000\000\000", td_fpop = 0x0, td_dbgflags = 0, td_dbgksi = {ksi_link = { tqe_next = 0x0, tqe_prev = 0x0}, ksi_info = {si_signo = 0, si_errno = 0, si_code = 0, si_pid = 0, si_uid = 0, si_status = 0, si_addr = 0x0, si_value = {sival_int = 0, sival_ptr = 0x0, sigval_int = 0, sigval_ptr = 0x0}, _reason = {_fault = {_trapno = 0}, _timer = {_timerid = 0, _overrun = 0}, _mesgq = { _mqd = 0}, _poll = {_band = 0}, __spare__ = {__spare1__ = 0, __spare2__ = {0, 0, 0, 0, 0, 0, 0}}}}, ksi_flags = 0, ksi_sigq = 0x0}, td_ng_outbound = 0, td_osd = {osd_nslots = 0, osd_slots = 0x0, osd_next = { le_next = 0x0, le_prev = 0x0}}, td_rqindex = 32 ' ', td_base_pri = 160 ' ', td_priority = 128 '\200', td_pri_class = 3 '\003', td_user_pri = 128 '\200', td_base_user_pri = 128 '\200', td_pcb = 0xe7d14d80, td_state = TDS_RUNNING, td_retval = {0, 0}, td_slpcallout = {c_links = {sle = {sle_next = 0xc5d704e0}, tqe = { tqe_next = 0xc5d704e0, tqe_prev = 0xc55bb1f0}}, c_time = 20246590, c_arg = 0xc5dac2e0, c_func = 0xc0736bc0 , c_lock = 0x0, c_flags = 18, c_cpu = 32}, td_frame = 0xe7d14d28, td_kstack_obj = 0xc6182088, td_kstack = 3889246208, td_kstack_pages = 2, td_unused1 = 0x0, td_unused2 = 0, td_unused3 = 0, td_critnest = 1, td_md = {md_spinlock_count = 1, md_saved_flags = 582}, td_sched = 0xc5dac58c, td_ar = 0x0, td_syscalls = 0, td_lprof = {{lh_first = 0x0}, {lh_first = 0x0}}, td_dtrace = 0x0, td_errno = 0, td_vnet = 0x0, td_vnet_lpush = 0x0, td_rux = {rux_runtime = 87350143265, rux_uticks = 0, rux_sticks = 5278, rux_iticks = 0, rux_uu = 0, rux_su = 0, rux_tu = 0}, td_map_def_user = 0x0, td_dbg_forked = 0} (kgdb) p *newtd $2 = {td_lock = 0xc0b1cb80, td_proc = 0xc553a810, td_plist = {tqe_next = 0xc553c8a0, tqe_prev = 0xc553a818}, td_runq = {tqe_next = 0x0, tqe_prev = 0x0}, td_slpq = {tqe_next = 0x0, tqe_prev = 0x0}, td_lockq = { tqe_next = 0x0, tqe_prev = 0x0}, td_cpuset = 0xc5533e38, td_sel = 0x0, td_sleepqueue = 0xc5531e00, td_turnstile = 0xc553d000, td_umtxq = 0xc5527ac0, td_tid = 100004, td_sigqueue = {sq_signals = {__bits = {0, 0, 0, 0}}, sq_kill = {__bits = {0, 0, 0, 0}}, sq_list = {tqh_first = 0x0, tqh_last = 0xc553c620}, sq_proc = 0xc553a810, sq_flags = 1}, td_flags = 262180, td_inhibitors = 0, td_pflags = 2097152, td_dupfd = 0, td_sqqueue = 0, td_wchan = 0x0, td_wmesg = 0x0, td_lastcpu = 0 '\0', td_oncpu = 255 'ÿ', td_owepreempt = 0 '\0', td_tsqueue = 0 '\0', td_locks = 0, td_rw_rlocks = 0, td_lk_slocks = 0, td_blocked = 0x0, td_lockname = 0x0, td_contested = {lh_first = 0x0}, td_sleeplocks = 0x0, td_intr_nesting_level = 0, td_pinned = 0, td_ucred = 0xc5535600, td_estcpu = 0, td_slptick = 0, td_blktick = 0, td_ru = {ru_utime = {tv_sec = 0, tv_usec = 0}, ru_stime = {tv_sec = 0, tv_usec = 0}, ru_maxrss = 0, ru_ixrss = 0, ru_idrss = 0, ru_isrss = 0, ru_minflt = 0, ru_majflt = 0, ru_nswap = 0, ru_inblock = 0, ru_oublock = 0, ru_msgsnd = 0, ru_msgrcv = 0, ru_nsignals = 0, ru_nvcsw = 33962290, ru_nivcsw = 40323696}, td_incruntime = 370201469879, td_runtime = 41685199750119, td_pticks = 2502607, td_sticks = 22282, td_iticks = 0, td_uticks = 0, td_intrval = 0, td_oldsigmask = {__bits = {0, 0, 0, 0}}, td_sigmask = {__bits = {0, 0, 0, 0}}, td_generation = 74285986, td_sigstk = {ss_sp = 0x0, ss_size = 0, ss_flags = 0}, td_xsig = 0, td_profil_addr = 0, td_profil_ticks = 0, td_name = "idle: cpu0\000\000\000\000\000\000\000\000\000", td_fpop = 0x0, td_dbgflags = 0, td_dbgksi = { ksi_link = {tqe_next = 0x0, tqe_prev = 0x0}, ksi_info = {si_signo = 0, si_errno = 0, si_code = 0, si_pid = 0, si_uid = 0, si_status = 0, si_addr = 0x0, si_value = {sival_int = 0, sival_ptr = 0x0, sigval_int = 0, sigval_ptr = 0x0}, _reason = {_fault = {_trapno = 0}, _timer = {_timerid = 0, _overrun = 0}, _mesgq = {_mqd = 0}, _poll = {_band = 0}, __spare__ = {__spare1__ = 0, __spare2__ = { 0, 0, 0, 0, 0, 0, 0}}}}, ksi_flags = 0, ksi_sigq = 0x0}, td_ng_outbound = 0, td_osd = { osd_nslots = 0, osd_slots = 0x0, osd_next = {le_next = 0x0, le_prev = 0x0}}, td_rqindex = 0 '\0', td_base_pri = 255 'ÿ', td_priority = 255 'ÿ', td_pri_class = 4 '\004', td_user_pri = 160 ' ', td_base_user_pri = 160 ' ', td_pcb = 0xc51e3d80, td_state = TDS_CAN_RUN, td_retval = {0, 0}, td_slpcallout = { c_links = {sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0x0}}, c_time = 0, c_arg = 0x0, c_func = 0, c_lock = 0x0, c_flags = 16, c_cpu = 0}, td_frame = 0xc51e3d28, td_kstack_obj = 0xc157ddd0, td_kstack = 3307085824, td_kstack_pages = 2, td_unused1 = 0x0, td_unused2 = 0, td_unused3 = 0, td_critnest = 1, td_md = {md_spinlock_count = 1, md_saved_flags = 582}, td_sched = 0xc553c86c, td_ar = 0x0, td_syscalls = 0, td_lprof = {{lh_first = 0x0}, {lh_first = 0x0}}, td_dtrace = 0x0, td_errno = 0, td_vnet = 0x0, td_vnet_lpush = 0x0, td_rux = {rux_runtime = 41315105445882, rux_uticks = 0, rux_sticks = 2480325, rux_iticks = 0, rux_uu = 0, rux_su = 0, rux_tu = 0}, td_map_def_user = 0x0, td_dbg_forked = 0} (kgdb) p *mtx $3 = {lock_object = {lo_name = 0xc0a3af04 "sleepq chain", lo_flags = 720896, lo_data = 0, lo_witness = 0x0}, mtx_lock = 4} (kgdb) disassemble Dump of assembler code for function sched_switch: 0xc07206c0 : push %ebp 0xc07206c1 : mov %esp,%ebp 0xc07206c3 : push %edi 0xc07206c4 : push %esi 0xc07206c5 : push %ebx 0xc07206c6 : sub $0x24,%esp 0xc07206c9 : mov 0x8(%ebp),%esi 0xc07206cc : mov (%esi),%eax 0xc07206ce : mov %fs:0x20,%eax 0xc07206d4 : mov %eax,0xfffffff0(%ebp) 0xc07206d7 : mov %eax,0xffffffec(%ebp) 0xc07206da : imul $0x680,%eax,%eax 0xc07206e0 : lea 0xc0b1cb80(%eax),%edi 0xc07206e6 : mov 0x248(%esi),%ebx 0xc07206ec : mov (%esi),%eax 0xc07206ee : mov %eax,0xffffffe4(%ebp) 0xc07206f1 : mov 0xc0b16e8c,%eax 0xc07206f6 : mov %eax,0x8(%ebx) 0xc07206f9 : movzbl 0x8d(%esi),%eax 0xc0720700 : mov %al,0x8c(%esi) 0xc0720706 : movb $0xff,0x8d(%esi) 0xc072070d : mov 0x10(%ebp),%eax 0xc0720710 : and $0x400,%eax 0xc0720715 : jne 0xc072071e 0xc0720717 : andl $0xfffeffff,0x70(%esi) 0xc072071e : movb $0x0,0x8e(%esi) 0xc0720725 : addw $0x1,0x24(%edi) 0xc072072a : testb $0x20,0x70(%esi) 0xc072072e : je 0xc0720740 0xc0720730 : movl $0x2,0x1f4(%esi) 0xc072073a : jmp 0xc0720924 0xc072073f : nop 0xc0720740 : cmpl $0x4,0x1f4(%esi) 0xc0720747 : jne 0xc07208c0 0xc072074d : cmp $0x1,%eax 0xc0720750 : sbb %edx,%edx 0xc0720752 : and $0xfffffff8,%edx 0xc0720755 : add $0xb,%edx 0xc0720758 : mov %edx,0xffffffe8(%ebp) 0xc072075b : cmpl $0x0,0xac(%esi) 0xc0720762 : jne 0xc0720790 0xc0720764 : mov 0x28(%esi),%edx 0xc0720767 : movzbl 0x6(%ebx),%ecx ---Type to continue, or q to quit--- 0xc072076b : mov %ecx,%eax 0xc072076d : shr $0x5,%al 0xc0720770 : movzbl %al,%eax 0xc0720773 : mov (%edx,%eax,4),%eax 0xc0720776 : and $0x1f,%ecx 0xc0720779 : sar %cl,%eax 0xc072077b : test $0x1,%al 0xc072077d : jne 0xc0720790 0xc072077f : mov $0x0,%edx 0xc0720784 : mov %esi,%eax 0xc0720786 : call 0xc071ff90 0xc072078b : mov %al,0x6(%ebx) 0xc072078e : mov %esi,%esi 0xc0720790 : movzbl 0x6(%ebx),%eax 0xc0720794 : cmp 0xffffffec(%ebp),%eax 0xc0720797 : jne 0xc0720850 0xc072079d : mov (%esi),%eax 0xc072079f : movzbl 0x1ea(%esi),%ebx 0xc07207a6 : mov 0x248(%esi),%ecx 0xc07207ac : movl $0x3,0x1f4(%esi) 0xc07207b6 : cmpl $0x0,0xac(%esi) 0xc07207bd : jne 0xc07207c8 0xc07207bf : addl $0x1,0x20(%edi) 0xc07207c3 : orw $0x2,0x4(%ecx) 0xc07207c8 : cmp $0x9f,%bl 0xc07207cb : ja 0xc07207d4 0xc07207cd : lea 0x2c(%edi),%eax 0xc07207d0 : mov %eax,(%ecx) 0xc07207d2 : jmp 0xc0720833 0xc07207d4 : cmp $0xdf,%bl 0xc07207d7 : ja 0xc072082b 0xc07207d9 : lea 0x234(%edi),%eax 0xc07207df : mov %eax,(%ecx) 0xc07207e1 : testb $0x18,0xffffffe8(%ebp) 0xc07207e5 : jne 0xc0720806 0xc07207e7 : movzbl 0x2a(%edi),%edx 0xc07207eb : lea 0x60(%ebx,%edx,1),%eax 0xc07207ef : and $0x3f,%eax 0xc07207f2 : movzbl 0x2b(%edi),%ebx 0xc07207f6 : cmp %dl,%bl 0xc07207f8 : je 0xc072080a 0xc07207fa : cmp %al,%bl 0xc07207fc : jne 0xc072080a 0xc07207fe : sub $0x1,%eax ---Type to continue, or q to quit--- 0xc0720801 : and $0x3f,%eax 0xc0720804 : jmp 0xc072080a 0xc0720806 : movzbl 0x2b(%edi),%eax 0xc072080a : movzbl %al,%eax 0xc072080d : mov (%ecx),%edx 0xc072080f : mov 0xffffffe8(%ebp),%ecx 0xc0720812 : mov %ecx,0xc(%esp) 0xc0720816 : mov %eax,0x8(%esp) 0xc072081a : mov %esi,0x4(%esp) 0xc072081e : mov %edx,(%esp) 0xc0720821 : call 0xc07051a0 0xc0720826 : jmp 0xc0720924 0xc072082b : lea 0x43c(%edi),%eax 0xc0720831 : mov %eax,(%ecx) 0xc0720833 : mov (%ecx),%eax 0xc0720835 : mov 0xffffffe8(%ebp),%edx 0xc0720838 : mov %edx,0x8(%esp) 0xc072083c : mov %esi,0x4(%esp) 0xc0720840 : mov %eax,(%esp) 0xc0720843 : call 0xc0704dc0 0xc0720848 : jmp 0xc0720924 0xc072084d : lea 0x0(%esi),%esi 0xc0720850 : mov 0x248(%esi),%eax 0xc0720856 : movzbl 0x6(%eax),%eax 0xc072085a : imul $0x680,%eax,%eax 0xc0720860 : lea 0xc0b1cb80(%eax),%ebx 0xc0720866 : mov %esi,%edx 0xc0720868 : mov %edi,%eax 0xc072086a : call 0xc071da30 0xc072086f : call 0xc09af910 0xc0720874 : mov %esi,(%esp) 0xc0720877 : call 0xc06edf20 0xc072087c : mov %edi,%edx 0xc072087e : mov %ebx,%eax 0xc0720880 : call 0xc071e080 0xc0720885 : mov 0xffffffe8(%ebp),%ecx 0xc0720888 : mov %esi,%edx 0xc072088a : mov %ebx,%eax 0xc072088c : call 0xc071ef50 0xc0720891 : mov %esi,%edx 0xc0720893 : mov %ebx,%eax 0xc0720895 : call 0xc071e990 0xc072089a : mov 0x8(%ebx),%eax 0xc072089d : test %eax,%eax ---Type to continue, or q to quit--- 0xc072089f : je 0xc07208a9 0xc07208a1 : sub $0x1,%eax 0xc07208a4 : mov %eax,0x8(%ebx) 0xc07208a7 : jmp 0xc07208b1 0xc07208a9 : mov $0x4,%eax 0xc07208ae : xchg %eax,0x10(%ebx) 0xc07208b1 : call 0xc09afae0 0xc07208b6 : call 0xc09afae0 0xc07208bb : mov %ebx,0xffffffe4(%ebp) 0xc07208be : jmp 0xc0720924 0xc07208c0 : mov %fs:0x0,%ebx 0xc07208c7 : call 0xc09af910 0xc07208cc : mov $0x4,%eax 0xc07208d1 : lock cmpxchg %ebx,0x10(%edi) 0xc07208d6 : sete %al 0xc07208d9 : test %al,%al 0xc07208db : jne 0xc0720910 0xc07208dd : mov 0x10(%edi),%eax 0xc07208e0 : cmp %ebx,%eax 0xc07208e2 : jne 0xc07208ea 0xc07208e4 : addl $0x1,0x8(%edi) 0xc07208e8 : jmp 0xc0720910 0xc07208ea : movl $0x0,0x10(%esp) 0xc07208f2 : movl $0x0,0xc(%esp) 0xc07208fa : movl $0x0,0x8(%esp) 0xc0720902 : mov %ebx,0x4(%esp) 0xc0720906 : mov %edi,(%esp) 0xc0720909 : call 0xc06ed970 <_mtx_lock_spin> 0xc072090e : mov %esi,%esi 0xc0720910 : mov %esi,(%esp) 0xc0720913 : call 0xc06edf20 0xc0720918 : mov %eax,0xffffffe4(%ebp) 0xc072091b : mov %esi,%edx 0xc072091d : mov %edi,%eax 0xc072091f : call 0xc071da30 0xc0720924 : call 0xc0705140 0xc0720929 : mov %eax,%ebx 0xc072092b : cmp %eax,%esi 0xc072092d : je 0xc07209b1 0xc0720933 : mov 0x4(%esi),%ecx 0xc0720936 : lock cmpxchg %eax,0x5c(%ecx) 0xc072093b : test $0x800000,%eax 0xc0720940 : je 0xc0720960 0xc0720942 : mov 0xc0b184dc,%eax ---Type to continue, or q to quit--- 0xc0720947 : test %eax,%eax 0xc0720949 : je 0xc0720960 0xc072094b : movl $0x0,0x8(%esp) 0xc0720953 : movl $0x3,0x4(%esp) 0xc072095b : mov %esi,(%esp) 0xc072095e : call *%eax 0xc0720960 : mov %ebx,0x10(%edi) 0xc0720963 : mov 0xffffffe4(%ebp),%edx 0xc0720966 : mov %edx,0x8(%esp) 0xc072096a : mov %ebx,0x4(%esp) 0xc072096e : mov %esi,(%esp) 0xc0720971 : call 0xc09bcdc4 0xc0720976 : mov %fs:0x20,%eax 0xc072097c : mov %eax,0xfffffff0(%ebp) 0xc072097f : mov %eax,0xffffffec(%ebp) 0xc0720982 : mov 0x4(%esi),%ecx 0xc0720985 : lock cmpxchg %eax,0x5c(%ecx) 0xc072098a : test $0x800000,%eax 0xc072098f : je 0xc07209b6 0xc0720991 : mov 0xc0b184dc,%eax 0xc0720996 : test %eax,%eax 0xc0720998 : je 0xc07209b6 0xc072099a : movl $0x0,0x8(%esp) 0xc07209a2 : movl $0x2,0x4(%esp) 0xc07209aa : mov %esi,(%esp) 0xc07209ad : call *%eax 0xc07209af : jmp 0xc07209b6 0xc07209b1 : mov 0xffffffe4(%ebp),%eax 0xc07209b4 : xchg %eax,(%esi) 0xc07209b6 : movzbl 0xffffffec(%ebp),%edx 0xc07209ba : mov %dl,0x8d(%esi) 0xc07209c0 : add $0x24,%esp 0xc07209c3 : pop %ebx 0xc07209c4 : pop %esi 0xc07209c5 : pop %edi 0xc07209c6 : pop %ebp 0xc07209c7 : ret End of assembler dump. -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/