From owner-freebsd-java@FreeBSD.ORG Thu May 25 14:09:45 2006 Return-Path: X-Original-To: freebsd-java@freebsd.org Delivered-To: freebsd-java@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B7D7B16A46F for ; Thu, 25 May 2006 14:09:45 +0000 (UTC) (envelope-from vadim_nuclight@mail.ru) Received: from mx7.mail.ru (mx7.mail.ru [194.67.23.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id CC59843D5C for ; Thu, 25 May 2006 14:09:34 +0000 (GMT) (envelope-from vadim_nuclight@mail.ru) Received: from [82.211.136.13] (port=34053 helo=nuclight.avtf.net) by mx7.mail.ru with esmtp id 1FjGWN-000Oaw-00 for freebsd-java@freebsd.org; Thu, 25 May 2006 18:09:04 +0400 To: freebsd-java@freebsd.org Date: Thu, 25 May 2006 21:06:59 +0700 From: "Vadim Goncharov" Organization: AVTF TPU Hostel Content-Type: text/plain; format=flowed; delsp=yes; charset=koi8-r MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-ID: User-Agent: Opera M2/7.54 (Win32, build 3865) Subject: diablo 1.5 died after 5 day uptime X-BeenThere: freebsd-java@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting Java to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 May 2006 14:09:45 -0000 Hello. I have an SMP (HTT) machine with 5.5-PRERELEASE, using official diablo 1.5 jdk binaries from beginning of April. As suggested, I now run java with -XX:+UseMembar cmdline switch, that resolved problem with crashing application after startup. But today after 5 days uptime now-debugged-and-set-to-long-run server process have crashed, with 100M coredump file. What happened? Below are log and gdb backtrace. As some additional info: I have limits in login.conf, 32M for resident size and 256M for virtual mem size; yesterday when I looked at process stats it was about 10M resident and 250M virtual - could be a reason?.. > more hs_err_pid54480.log # # An unexpected error has been detected by HotSpot Virtual Machine: # # SIGSEGV (0xb) at pc=0x0a3fcad9, pid=54480, tid=0x8a28000 # # Java VM: Java HotSpot(TM) Client VM (diablo-1.5.0_06-b00 mixed mode) # Problematic frame: # V [libjvm.so+0x26dad9] # --------------- T H R E A D --------------- Current thread is native thread siginfo:si_signo=11, si_errno=0, si_code=12, si_addr=0x00000000 Registers: EAX=0x00000000, EBX=0x0a612d98, ECX=0x00000000, EDX=0x08a28000 ESP=0xbf42ac54, EBP=0xbf42ac78, ESI=0x00000000, EDI=0x080600c0 EIP=0x0a3fcad9, EFLAGS=0x00010292 Top of Stack: (sp=0xbf42ac54) 0xbf42ac54: 000000fd 0a5e8380 0a5e8380 080600c0 0xbf42ac64: 0a5ccb85 00000001 0a3fca8a 0a612d98 0xbf42ac74: 080600c0 bf42ac88 0a3fcb36 ffffffff 0xbf42ac84: 0a612d98 bf42ad58 0a3fceda 00000800 0xbf42ac94: 080600c0 080600c0 080600c0 080600c0 0xbf42aca4: 080600c0 080600c0 0a179bac 0a608a78 0xbf42acb4: 0a5e26d7 0a608a78 0a608a78 0a5f7ce9 0xbf42acc4: 0a608a78 0a5d8c08 0a608a78 bf42ad20 Instructions: (pc=0x0a3fcad9) 0x0a3fcac9: 50 52 51 e8 3f d0 e9 ff 83 c4 10 89 c6 83 ec 0c 0x0a3fcad9: 8b 06 56 ff 50 14 83 c4 10 85 c0 74 0a c7 86 e8 Stack: [0xbf3eb000,0xbf42b000), sp=0xbf42ac54, free space=255k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x26dad9] V [libjvm.so+0x26db36] V [libjvm.so+0x26deda] V [libjvm.so+0x3841e0] V [libjvm.so+0x384216] V [libjvm.so+0x3f937d] V [libjvm.so+0x3ef88c] V [libjvm.so+0x3f6375] V [libjvm.so+0x3840a9] C [libpthread.so.1+0x8b09] _pthread_create+0x7c1 --------------- P R O C E S S --------------- Java Threads: ( => current thread ) 0x083f0e00 JavaThread "Thread-258253" [_thread_new, id=144867328] 0x083f0400 JavaThread "Thread-258251" [_thread_in_native, id=138726400] 0x08410600 JavaThread "Thread-258250" [_thread_in_native, id=144340480] 0x08310c00 JavaThread "Thread-258249" [_thread_in_native, id=138987520] 0x083c3600 JavaThread "Thread-257158" [_thread_in_native, id=138228736] 0x083c3a00 JavaThread "Thread-256615" [_thread_in_native, id=144158208] 0x08310e00 JavaThread "Thread-256480" [_thread_in_native, id=139173888] 0x08310400 JavaThread "Thread-255767" [_thread_in_native, id=138289152] 0x08310200 JavaThread "Thread-254463" [_thread_in_native, id=140458496] 0x08310000 JavaThread "Thread-253261" [_thread_in_native, id=140185600] 0x081a6c00 JavaThread "Thread-253072" [_thread_in_native, id=138986496] 0x083f0000 JavaThread "Thread-251582" [_thread_in_native, id=138600448] 0x08420400 JavaThread "Thread-251176" [_thread_in_native, id=144339968] 0x083c3400 JavaThread "Thread-249486" [_thread_in_native, id=138415616] 0x083c3200 JavaThread "Thread-249101" [_thread_in_native, id=138723328] 0x08410e00 JavaThread "Thread-247705" [_thread_in_native, id=135250432] 0x08410a00 JavaThread "Thread-246215" [_thread_in_native, id=138543104] 0x08410400 JavaThread "Thread-245829" [_thread_in_native, id=138667520] 0x083e2e00 JavaThread "Thread-245769" [_thread_in_native, id=139823104] 0x080fc400 JavaThread "Thread-244253" [_thread_in_native, id=144342016] 0x083d3a00 JavaThread "Thread-243747" [_thread_in_native, id=144935936] 0x08410200 JavaThread "Thread-242603" [_thread_in_native, id=138164224] 0x083e2600 JavaThread "Thread-242051" [_thread_in_native, id=140848128] 0x083e2a00 JavaThread "Thread-240936" [_thread_in_native, id=139108352] 0x083e2400 JavaThread "Thread-240760" [_thread_in_native, id=138347008] 0x083e2200 JavaThread "Thread-240659" [_thread_in_native, id=138988544] 0x081edc00 JavaThread "Thread-240643" [_thread_in_native, id=143752192] 0x081eda00 JavaThread "Thread-238918" [_thread_in_native, id=138229760] 0x083d3600 JavaThread "Thread-238834" [_thread_in_native, id=140126208] 0x0817ce00 JavaThread "Thread-238811" [_thread_in_native, id=138545152] 0x080fc800 JavaThread "Thread-238463" [_thread_in_native, id=138665984] 0x0817f600 JavaThread "Thread-235727" [_thread_in_native, id=140396544] 0x083d3200 JavaThread "Thread-233606" [_thread_in_native, id=140124160] 0x08310800 JavaThread "Thread-217066" [_thread_in_native, id=144745984] 0x08400400 JavaThread "Thread-215790" [_thread_in_native, id=144801792] 0x08410000 JavaThread "Thread-207532" [_thread_in_native, id=144933376] 0x083d3c00 JavaThread "Thread-205656" [_thread_in_native, id=138291200] 0x0817fe00 JavaThread "Thread-183584" [_thread_in_native, id=138603520] 0x083f0c00 JavaThread "Thread-164930" [_thread_in_native, id=138348032] 0x0805cc00 JavaThread "DestroyJavaVM" [_thread_blocked, id=134594560] 0x081a6600 JavaThread "Thread-3" [_thread_in_native, id=135948288] 0x081a6200 JavaThread "Thread-2" [_thread_blocked, id=135947264] 0x08160e00 JavaThread "Thread-1" [_thread_in_native, id=135946240] 0x08160a00 JavaThread "Thread-0" [_thread_in_native, id=135662592] 0x08143c00 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=135544320] 0x08143800 JavaThread "CompilerThread0" daemon [_thread_blocked, id=135543296] 0x08143400 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=135542272] 0x08143000 JavaThread "Finalizer" daemon [_thread_blocked, id=135541248] 0x08067c00 JavaThread "Reference Handler" daemon [_thread_blocked, id=134643200] Other Threads: 0x08056f00 VMThread [id=134642176] 0x08056a00 WatcherThread [id=135660032] VM state:not at safepoint (normal execution) VM Mutex/Monitor currently owned by a thread: None Heap def new generation total 576K, used 426K [0x0f5e0000, 0x0f680000, 0x0fac0000) eden space 512K, 82% used [0x0f5e0000, 0x0f649f60, 0x0f660000) from space 64K, 4% used [0x0f670000, 0x0f670ab8, 0x0f680000) to space 64K, 0% used [0x0f660000, 0x0f660000, 0x0f670000) tenured generation total 1408K, used 357K [0x0fac0000, 0x0fc20000, 0x135e0000) the space 1408K, 25% used [0x0fac0000, 0x0fb19520, 0x0fb19600, 0x0fc20000) compacting perm gen total 8192K, used 1777K [0x135e0000, 0x13de0000, 0x175e0000) the space 8192K, 21% used [0x135e0000, 0x1379c608, 0x1379c800, 0x13de0000) No shared spaces configured. Dynamic libraries: 0x08048000 /usr/local/diablo-jdk1.5.0/bin/java 0x0a07f000 /lib/libz.so.2 0x0a08f000 /usr/lib/libpthread.so.1 0x0a0b3000 /lib/libc.so.5 0x0a18f000 /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so 0x0a631000 /lib/libm.so.3 0x0a64d000 /usr/local/diablo-jdk1.5.0/jre/lib/i386/native_threads/libhpi.so 0x0a65e000 /usr/local/diablo-jdk1.5.0/jre/lib/i386/libverify.so 0x0a66b000 /usr/local/diablo-jdk1.5.0/jre/lib/i386/libjava.so 0x0a68a000 /usr/local/diablo-jdk1.5.0/jre/lib/i386/libzip.so 0x1774a000 /usr/local/diablo-jdk1.5.0/jre/lib/i386/libnet.so 0x0a052000 /libexec/ld-elf.so.1 VM Arguments: jvm_args: -XX:+UseMembar java_command: ServSender Launcher Type: SUN_STANDARD Environment Variables: JAVA_HOME=/usr/local/diablo-jdk1.5.0 PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:/home/ krot/bin LD_LIBRARY_PATH=/usr/local/diablo-jdk1.5.0/jre/lib/i386/client:/usr/local/diablo-jdk1.5.0/jre/lib/ i386:/usr/local/diablo-jdk1.5.0/jre/../lib/i386 SHELL=/usr/local/bin/bash Signal Handlers: SIGSEGV: [libjvm.so+0x426350], sa_mask[0]=0xffffffff, sa_flags=0x00000002 SIGBUS: [libjvm.so+0x426350], sa_mask[0]=0xffffffff, sa_flags=0x00000002 SIGFPE: [libjvm.so+0x386f00], sa_mask[0]=0xffffffff, sa_flags=0x00000042 SIGPIPE: [libjvm.so+0x386f00], sa_mask[0]=0xffffffff, sa_flags=0x00000042 SIGILL: [libjvm.so+0x386f00], sa_mask[0]=0xffffffff, sa_flags=0x00000042 SIGUSR1: [libjvm.so+0x386fd0], sa_mask[0]=0x00000000, sa_flags=0x00000040 SIGUSR2: [libjvm.so+0x386f00], sa_mask[0]=0xffffffff, sa_flags=0x00000042 SIGHUP: [libjvm.so+0x3857b0], sa_mask[0]=0xffffffff, sa_flags=0x00000002 SIGINT: [libjvm.so+0x3857b0], sa_mask[0]=0xffffffff, sa_flags=0x00000002 SIGQUIT: [libjvm.so+0x3857b0], sa_mask[0]=0xffffffff, sa_flags=0x00000002 SIGTERM: [libjvm.so+0x3857b0], sa_mask[0]=0xffffffff, sa_flags=0x00000002 --------------- S Y S T E M --------------- OS:FreeBSD uname:FreeBSD 5.5-PRERELEASE FreeBSD 5.5-PRERELEASE #1: Sun May 7 19:09:52 NOVST 2006 vadim@h ostel.avtf.net:/usr/obj/usr/src/sys/HOSTEL i386 rlimit: STACK 8192k, CORE 102400k, NOFILE 256 CPU:total 2 family 15, cmov, cx8, fxsr, mmx, ht Memory: 4k page, physical 171860k vm_info: Java HotSpot(TM) Client VM (diablo-1.5.0_06-b00) for freebsd-x86, built on Feb 16 2006 13 :17:48 by truk with gcc 3.4.2 [FreeBSD] 20040728 > gdb java.core /usr/local/diablo-jdk1.5.0/bin/java GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"..."/home/krot/ServSender/java.core": not in executable format: File format not recognized "/usr/local/diablo-jdk1.5.0/bin/java" is not a core dump: File format not recognized (gdb) Quit hostel:~# gdb /usr/local/diablo-jdk1.5.0/bin/java ~krot/ServSender/java.core 0/p0 [21:01:12] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"...(no debugging symbols found)... Core was generated by `java'. Program terminated with signal 6, Aborted. Reading symbols from /lib/libz.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libz.so.2 Reading symbols from /usr/lib/libpthread.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libpthread.so.1 Reading symbols from /lib/libc.so.5...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.5 Reading symbols from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so Reading symbols from /lib/libm.so.3...(no debugging symbols found)...done. Loaded symbols for /lib/libm.so.3 Reading symbols from /usr/local/diablo-jdk1.5.0/jre/lib/i386/native_threads/libhpi.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/diablo-jdk1.5.0/jre/lib/i386/native_threads/libhpi.so Reading symbols from /usr/local/diablo-jdk1.5.0/jre/lib/i386/libverify.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/diablo-jdk1.5.0/jre/lib/i386/libverify.so Reading symbols from /usr/local/diablo-jdk1.5.0/jre/lib/i386/libjava.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/diablo-jdk1.5.0/jre/lib/i386/libjava.so Reading symbols from /usr/local/diablo-jdk1.5.0/jre/lib/i386/libzip.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/diablo-jdk1.5.0/jre/lib/i386/libzip.so Reading symbols from /usr/local/diablo-jdk1.5.0/jre/lib/i386/libnet.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/diablo-jdk1.5.0/jre/lib/i386/libnet.so Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x0a0ab373 in pthread_testcancel () from /usr/lib/libpthread.so.1 (gdb) bt #0 0x0a0ab373 in pthread_testcancel () from /usr/lib/libpthread.so.1 #1 0x0a09c171 in sigaction () from /usr/lib/libpthread.so.1 #2 0x0a0961f5 in pthread_kill () from /usr/lib/libpthread.so.1 #3 0x0a095bc4 in raise () from /usr/lib/libpthread.so.1 #4 0x0a168cef in abort () from /lib/libc.so.5 #5 0x0a513512 in os::abort () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #6 0x0a5b4b00 in VMError::report_and_die () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #7 0x0a5174ad in JVM_handle_bsd_signal () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #8 0x0a515f24 in signalHandler () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #9 0x0a09ae37 in sigaction () from /usr/lib/libpthread.so.1 #10 0x0a09acd7 in sigaction () from /usr/lib/libpthread.so.1 #11 0x0a09b898 in sigaction () from /usr/lib/libpthread.so.1 #12 0x0a0a40c0 in pthread_mutexattr_init () from /usr/lib/libpthread.so.1 #13 0x0a0a3fb4 in pthread_mutexattr_init () from /usr/lib/libpthread.so.1 #14 0x0a154327 in _ctx_start () from /lib/libc.so.5 #15 0x00000000 in ?? () #16 0xbf42ac10 in ?? () #17 0xbf42a950 in ?? () #18 0x00000000 in ?? () #19 0x0a0a3f50 in pthread_mutexattr_init () from /usr/lib/libpthread.so.1 #20 0x0a3fcb36 in vm_abort () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #21 0x0a3fceda in vm_exit_out_of_memory () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #22 0x080600c0 in ?? () #23 0x080600c0 in ?? () #24 0x080600c0 in ?? () #25 0x080600c0 in ?? () #26 0x080600c0 in ?? () #27 0x080600c0 in ?? () #28 0x0a179bac in ?? () from /lib/libc.so.5 #29 0x0a608a78 in vtable for defaultStream () ---Type to continue, or q to quit--- from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #30 0x0a5e26d7 in GenCollectedHeap::preload_and_dump(Thread*)::map_entry_array_sig () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #31 0x0a608a78 in vtable for defaultStream () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #32 0x0a608a78 in vtable for defaultStream () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #33 0x0a5f7ce9 in standard_subs () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #34 0x0a608a78 in vtable for defaultStream () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #35 0x0a5d8c08 in pi_4 () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #36 0x0a608a78 in vtable for defaultStream () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #37 0xbf42ad20 in ?? () #38 0x0a5e266c in GenCollectedHeap::preload_and_dump(Thread*)::map_entry_array_sig () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #39 0x0a627390 in gclog_or_tty () from /usr/local/diablo-jdk1.5.0/jre/lib/i386/client/libjvm.so #40 0x00000001 in ?? () #41 0x0a0ae7a4 in _thread_state_running () from /usr/lib/libpthread.so.1 #42 0x00000000 in ?? () #43 0x0a0a74b1 in pthread_mutexattr_init () from /usr/lib/libpthread.so.1 Previous frame inner to this frame (corrupt stack?) (gdb) -- WBR, Vadim Goncharov