Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 29 Oct 2012 13:17:10 -0700
From:      Greg Lewis <glewis@eyesbeyond.com>
To:        Valentin Popov <valentin.po@gmail.com>
Cc:        freebsd-java@freebsd.org
Subject:   Re: [libnet.so+0x920a] Java_java_net_SocketInputStream_socketRead0+0x9a
Message-ID:  <20121029201710.GA12518@misty.eyesbeyond.com>
In-Reply-To: <F6779190-135D-490C-AF45-D60FA3015E0B@gmail.com>
References:  <3B7D5B68-C1EE-430B-B07F-D6814B81BD26@gmail.com> <20121029013759.GA2710@misty.eyesbeyond.com> <F6779190-135D-490C-AF45-D60FA3015E0B@gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
G'day Valentin,

On Mon, Oct 29, 2012 at 05:50:35PM +0400, Valentin Popov wrote:
> Another one on different server, 
> 
> if you need a can save core?  

It seems like you're saying that under load you get random segfaults that
occur in different places in the JVM.  Those kind of problems generally
take a bit of tracking down compared to something that happens reliably
and/or happens in the same place.

A core would be helpful if I had an amd64 machine to look at it on.  Let me
see if I can get to one in the FreeBSD cluster first.

> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00000008d871849a, pid=65220, tid=38136644544
> #
> # JRE version: 7.0-b24
> # Java VM: OpenJDK 64-Bit Server VM (23.2-b09 mixed mode bsd-amd64 compressed oops)
> # Problematic frame:
> # C  [libnet.so+0x949a]  Java_java_net_SocketInputStream_socketRead0+0x9a
> #
> # Core dump written. Default location: /usr/local/mailarchiva/server/logs/core or core.65220
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.sun.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
> 
> ---------------  T H R E A D  ---------------
> 
> Current thread (0x0000000800d52800):  JavaThread "RMI TCP Connection(2884)-192.168.1.2" daemon [_thread_in_native, id=-518061120, stack(0x00007fffe363c000,0x00007fffe373c000)]
> 
> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00000000000004d8
> 
> Registers:
> RAX=0x0000000000000007, RBX=0x0000000000000007, RCX=0x0000000000000000, RDX=0x0000000800c041c0
> RSP=0x00007fffe372b4e0, RBP=0x00007fffe373b530, RSI=0x00000008d6b78ba0, RDI=0x00000008d6b31ca0
> R8 =0x0000000000000000, R9 =0x0000000000000000, R10=0x0000000801a19780, R11=0x00007fffe373b628
> R12=0x00000000000006ca, R13=0x00000000000004d8, R14=0x00007fffe372b500, R15=0x00000000006ddd00
> RIP=0x00000008d871849a, EFLAGS=0x0000000000000001, ERR=0x0000000000000004
>   TRAPNO=0x000000000000000c
> 
> Top of Stack: (sp=0x00007fffe372b4e0)
> 0x00007fffe372b4e0:   2e61766100000000 00007fffe372b500
> 0x00007fffe372b4f0:   0000000000000000 00007fffe373b558
> 0x00007fffe372b500:   6e4b0200494d524a 656d697400000000
> 0x00007fffe372b510:   7571696e00000000 0000028000000000
> 0x00007fffe372b520:   685fd84700000000 00007fff00000000
> 0x00007fffe372b530:   0000000800000000 0000067700000000
> 0x00007fffe372b540:   ad001e2000000000 7a61ccf800000000
> 0x00007fffe372b550:   fa3e953100000000 ddfe94a400000000
> 0x00007fffe372b560:   010317e300000000 e53ca9b200000000
> 0x00007fffe372b570:   5f6a252200000000 dbe7e0c700000000
> 0x00007fffe372b580:   2169e66300000000 2da845b000000000
> 0x00007fffe372b590:   758f3213000003b0 0000000800000000
> 0x00007fffe372b5a0:   0000000000000000 00007fff00000000
> 0x00007fffe372b5b0:   0000018200000000 00007fff00000000
> 0x00007fffe372b5c0:   0022770500000000 0002000000000000
> 0x00007fffe372b5d0:   0000000000000000 0000000000000000
> 0x00007fffe372b5e0:   86f28b8d00000000 6a4c5b1800000000
> 0x00007fffe372b5f0:   2e696d7200000000 4f2e726500000000
> 0x00007fffe372b600:   0013873b00000000 0000027e00000000
> 0x00007fffe372b610:   7700000000000000 0b00000000000000
> 0x00007fffe372b620:   76616a1200000000 67642e6900000000
> 0x00007fffe372b630:   b065736100000000 0234dc4a00000000
> 0x00007fffe372b640:   6c61760500000000 696d760400000000
> 0x00007fffe372b650:   76616a4c00000000 67642f6900000000
> 0x00007fffe372b660:   703b444900000000 0900000000000000
> 0x00007fffe372b670:   616a110000000000 642e696d00000000
> 0x00007fffe372b680:   f844494d00000000 02b66da500000000
> 0x00007fffe372b690:   64646104000003b0 004c425b00000000
> 0x00007fffe372b6a0:   4c15007400000000 696d722f00000000
> 0x00007fffe372b6b0:   2f72657600000000 7570787000000008
> 0x00007fffe372b6c0:   17f3ac4200000000 000002e000000000
> 0x00007fffe372b6d0:   f008000000000000 738876b000000000 
> 
> Instructions: (pc=0x00000008d871849a)
> 0x00000008d871847a:   89 c6 45 85 ff 75 61 48 63 d3 4c 89 f6 44 89 e7
> 0x00000008d871848a:   e8 8d c1 ff ff 83 f8 00 89 c3 0f 8e 8c 00 00 00
> 0x00000008d871849a:   49 8b 45 00 4d 89 f0 89 d9 8b 95 c4 ff fe ff 48
> 0x00000008d87184aa:   8b b5 c8 ff fe ff 4c 89 ef ff 90 80 06 00 00 4c 
> 
> Register to memory mapping:
> 
> RAX=0x0000000000000007 is an unknown value
> RBX=0x0000000000000007 is an unknown value
> RCX=0x0000000000000000 is an unknown value
> RDX=0x0000000800c041c0 is an unknown value
> RSP=0x00007fffe372b4e0 is pointing into the stack for thread: 0x0000000800d52800
> RBP=0x00007fffe373b530 is pointing into the stack for thread: 0x0000000800d52800
> RSI=0x00000008d6b78ba0 is an unknown value
> RDI=0x00000008d6b31ca0 is an unknown value
> R8 =0x0000000000000000 is an unknown value
> R9 =0x0000000000000000 is an unknown value
> R10=0x0000000801a19780: +0xf6c0 in /usr/local/openjdk7/jre/lib/amd64/server/libjvm.so at 0x0000000800e00000
> R11=0x00007fffe373b628 is pointing into the stack for thread: 0x0000000800d52800
> R12=0x00000000000006ca is an unknown value
> R13=0x00000000000004d8 is an unknown value
> R14=0x00007fffe372b500 is pointing into the stack for thread: 0x0000000800d52800
> R15=0x00000000006ddd00 is an unknown value
> 
> 
> Stack: [0x00007fffe363c000,0x00007fffe373c000],  sp=0x00007fffe372b4e0,  free space=957k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> C  [libnet.so+0x949a]  Java_java_net_SocketInputStream_socketRead0+0x9a
> J  java.net.SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I
> 
> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
> J  java.net.SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I
> J  java.net.SocketInputStream.read([BIII)I
> J  java.net.SocketInputStream.read([BII)I
> J  java.io.BufferedInputStream.fill()V
> J  java.io.BufferedInputStream.read()I
> j  java.io.DataInputStream.readInt()I+4
> j  sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0()V+100
> j  sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run()V+50
> J  java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V
> j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
> j  java.lang.Thread.run()V+11
> v  ~StubRoutines::call_stub
> 
> ---------------  P R O C E S S  ---------------
> 
> Java Threads: ( => current thread )
> =>0x0000000800d52800 JavaThread "RMI TCP Connection(2884)-192.168.1.2" daemon [_thread_in_native, id=-518061120, stack(0x00007fffe363c000,0x00007fffe373c000)]
>   0x00000008e3048800 JavaThread "keepalive 192.168.1.10" daemon [_thread_blocked, id=-494774592, stack(0x00007fffe7379000,0x00007fffe7479000)]
>   0x00000008da2da800 JavaThread "net.java.truecommons.io.Streams$ReaderThread" daemon [_thread_blocked, id=-571300672, stack(0x00007fffe373d000,0x00007fffe383d000)]
>   0x00000008ec223000 JavaThread "ActiveMQ VMTransport: vm://ROOT#1-4844" daemon [_thread_blocked, id=-621405312, stack(0x00007fffe393f000,0x00007fffe3a3f000)]
>   0x00000008e6bc7000 JavaThread "net.java.truecommons.io.Streams$ReaderThread" daemon [_thread_blocked, id=-497652800, stack(0x00007fffea6ac000,0x00007fffea7ac000)]
>   0x00000008e5b95000 JavaThread "ActiveMQ Session Task-4964" [_thread_blocked, id=-497642944, stack(0x00007fffe6369000,0x00007fffe6469000)]
>   0x00000008eaa64800 JavaThread "ActiveMQ BrokerService[ROOT] Task-9310" daemon [_thread_blocked, id=-497648320, stack(0x00007ffff474c000,0x00007ffff484c000)]
>   0x00000008ec223800 JavaThread "ActiveMQ VMTransport: vm://ROOT#0-3976" daemon [_thread_blocked, id=-666082560, stack(0x00007fffe454b000,0x00007fffe464b000)]
>   0x00000008e6bd8800 JavaThread "ActiveMQ VMTransport: vm://ROOT#1-4843" daemon [_thread_blocked, id=-666084800, stack(0x00007ffff373c000,0x00007ffff383c000)]
>   0x00000008e82f0000 JavaThread "ActiveMQ BrokerService[ROOT] Task-9306" daemon [_thread_blocked, id=-494772352, stack(0x00007fffe4349000,0x00007fffe4449000)]
>   0x00000008e85ad800 JavaThread "JavaMail-EventQueue" daemon [_thread_blocked, id=-314463616, stack(0x00007fffe343a000,0x00007fffe353a000)]
>   0x00000008e365b000 JavaThread "_opt_soaps_store_7 committer" daemon [_thread_blocked, id=-478540352, stack(0x00007fffe474d000,0x00007fffe484d000)]
>   0x00000008e365c000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-478538112, stack(0x00007fffe484e000,0x00007fffe494e000)]
>   0x00000008e365c800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-478535872, stack(0x00007fffe494f000,0x00007fffe4a4f000)]
>   0x00000008e365d800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-478533632, stack(0x00007fffe4a50000,0x00007fffe4b50000)]
>   0x00000008e365e000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-478531392, stack(0x00007fffe4b51000,0x00007fffe4c51000)]
>   0x00000008e365f000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-478529152, stack(0x00007fffe4c52000,0x00007fffe4d52000)]
>   0x00000008e365f800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-606325312, stack(0x00007fffe4d53000,0x00007fffe4e53000)]
>   0x00000008e3660800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-606323072, stack(0x00007fffe4e54000,0x00007fffe4f54000)]
>   0x00000008e3661000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-606320832, stack(0x00007fffe4f55000,0x00007fffe5055000)]
>   0x00000008e3662000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-478621248, stack(0x00007fffe5056000,0x00007fffe5156000)]
>   0x00000008e3662800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-478619008, stack(0x00007fffe5157000,0x00007fffe5257000)]
>   0x00000008e3663800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-478616768, stack(0x00007fffe5258000,0x00007fffe5358000)]
>   0x00000008e3665000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-478614528, stack(0x00007fffe5359000,0x00007fffe5459000)]
>   0x00000008e3665800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-478612288, stack(0x00007fffe545a000,0x00007fffe555a000)]
>   0x00000008e3666800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-478610048, stack(0x00007fffe555b000,0x00007fffe565b000)]
>   0x00000008e3667000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-478607808, stack(0x00007fffe565c000,0x00007fffe575c000)]
>   0x00000008e3668000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-549732928, stack(0x00007ffff4146000,0x00007ffff4246000)]
>   0x00000008e3668800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517428672, stack(0x00007ffff2b30000,0x00007ffff2c30000)]
>   0x00000008e3669800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517620992, stack(0x00007ffff3237000,0x00007ffff3337000)]
>   0x00000008e366a000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517448832, stack(0x00007ffff3b40000,0x00007ffff3c40000)]
>   0x00000008e366b000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517421952, stack(0x00007ffff2e33000,0x00007ffff2f33000)]
>   0x00000008e366b800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517417472, stack(0x00007ffff3035000,0x00007ffff3135000)]
>   0x00000008e366c800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517440320, stack(0x00007ffff262b000,0x00007ffff272b000)]
>   0x00000008e366d000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517438080, stack(0x00007ffff272c000,0x00007ffff282c000)]
>   0x00000008e366e000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517430912, stack(0x00007ffff2a2f000,0x00007ffff2b2f000)]
>   0x00000008e366e800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517462272, stack(0x00007ffff353a000,0x00007ffff363a000)]
>   0x00000008e366f800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517419712, stack(0x00007ffff2f34000,0x00007ffff3034000)]
>   0x00000008e3671000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-549730688, stack(0x00007ffff4247000,0x00007ffff4347000)]
>   0x00000008e3671800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-549737408, stack(0x00007ffff3f44000,0x00007ffff4044000)]
>   0x00000008e3672800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517460032, stack(0x00007ffff363b000,0x00007ffff373b000)]
>   0x00000008e3673000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517616512, stack(0x00007ffff3439000,0x00007ffff3539000)]
>   0x00000008e3674000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517292288, stack(0x00007ffff2328000,0x00007ffff2428000)]
>   0x00000008e3674800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517435840, stack(0x00007ffff282d000,0x00007ffff292d000)]
>   0x00000008e3675800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-549726208, stack(0x00007ffff4449000,0x00007ffff4549000)]
>   0x00000008e3676000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-549739648, stack(0x00007ffff3e43000,0x00007ffff3f43000)]
>   0x00000008e3677000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517446592, stack(0x00007ffff3c41000,0x00007ffff3d41000)]
>   0x00000008e3677800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517296768, stack(0x00007ffff2126000,0x00007ffff2226000)]
>   0x00000008e3678800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517433152, stack(0x00007ffff292e000,0x00007ffff2a2e000)]
>   0x00000008e3679000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517426432, stack(0x00007ffff2c31000,0x00007ffff2d31000)]
>   0x00000008e367a000 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517290048, stack(0x00007ffff2429000,0x00007ffff2529000)]
>   0x00000008e367a800 JavaThread "_opt_soaps_store_7 indexers" daemon [_thread_blocked, id=-517451072, stack(0x00007ffff3a3f000,0x00007ffff3b3f000)]
>   0x00000008dc12d800 JavaThread "_opt_soaps_store_6 committer" daemon [_thread_blocked, id=-606335616, stack(0x00007fffe757b000,0x00007fffe767b000)]
>   0x00000008ecfcc000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-606333376, stack(0x00007fffe767c000,0x00007fffe777c000)]
>   0x00000008ecfcc800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-606331136, stack(0x00007fffe777d000,0x00007fffe787d000)]
>   0x00000008ecfcd800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-606328896, stack(0x00007fffe787e000,0x00007fffe797e000)]
>   0x00000008ecfce000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-608033792, stack(0x00007fffe797f000,0x00007fffe7a7f000)]
>   0x00000008ecfcf000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-608031552, stack(0x00007fffe7a80000,0x00007fffe7b80000)]
>   0x00000008ecfcf800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-608029312, stack(0x00007fffe7b81000,0x00007fffe7c81000)]
>   0x00000008ecfd0800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-608027072, stack(0x00007fffe7c82000,0x00007fffe7d82000)]
>   0x00000008ecfd1000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-602416960, stack(0x00007fffe7d83000,0x00007fffe7e83000)]
>   0x00000008ecfd2000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-602414720, stack(0x00007fffe7e84000,0x00007fffe7f84000)]
>   0x00000008ecfd2800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-602412480, stack(0x00007fffe7f85000,0x00007fffe8085000)]
>   0x00000008ecfd3800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-602410240, stack(0x00007fffe8086000,0x00007fffe8186000)]
>   0x00000008ecfd4000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-602408000, stack(0x00007fffe8187000,0x00007fffe8287000)]
>   0x00000008ecfd5000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-602405760, stack(0x00007fffe8288000,0x00007fffe8388000)]
>   0x00000008ecfd5800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-602403520, stack(0x00007fffe8389000,0x00007fffe8489000)]
>   0x00000008ecfd6800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-602401280, stack(0x00007fffe848a000,0x00007fffe858a000)]
>   0x00000008ecfd8000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-602399040, stack(0x00007fffe858b000,0x00007fffe868b000)]
>   0x00000008ecfd8800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-609207552, stack(0x00007fffe868c000,0x00007fffe878c000)]
>   0x00000008ecfd9800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-609205312, stack(0x00007fffe878d000,0x00007fffe888d000)]
>   0x00000008ecfda000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-609203072, stack(0x00007fffe888e000,0x00007fffe898e000)]
>   0x00000008ecfdb000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-608053504, stack(0x00007fffe898f000,0x00007fffe8a8f000)]
>   0x00000008ecfdb800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-608051264, stack(0x00007fffe8a90000,0x00007fffe8b90000)]
>   0x00000008ecfdc800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-608049024, stack(0x00007fffe8b91000,0x00007fffe8c91000)]
>   0x00000008ecfdd000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-608046784, stack(0x00007fffe8c92000,0x00007fffe8d92000)]
>   0x00000008ecfde000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-608044544, stack(0x00007fffe8d93000,0x00007fffe8e93000)]
>   0x00000008ecfde800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-608042304, stack(0x00007fffe8e94000,0x00007fffe8f94000)]
>   0x00000008ecfdf800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-608040064, stack(0x00007fffe8f95000,0x00007fffe9095000)]
>   0x00000008ecfe0000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-608037824, stack(0x00007fffe9096000,0x00007fffe9196000)]
>   0x00000008ecfe1000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-608035584, stack(0x00007fffe9197000,0x00007fffe9297000)]
>   0x00000008ecfe1800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-650618560, stack(0x00007fffe9298000,0x00007fffe9398000)]
>   0x00000008ecfe2800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-650616320, stack(0x00007fffe9399000,0x00007fffe9499000)]
>   0x00000008ecfe4000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-650614080, stack(0x00007fffe949a000,0x00007fffe959a000)]
>   0x00000008ecfe4800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-609229504, stack(0x00007fffe959b000,0x00007fffe969b000)]
>   0x00000008ecfe5800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-609227264, stack(0x00007fffe969c000,0x00007fffe979c000)]
>   0x00000008ecfe6000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-609225024, stack(0x00007fffe979d000,0x00007fffe989d000)]
>   0x00000008ecfe7000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-609222784, stack(0x00007fffe989e000,0x00007fffe999e000)]
>   0x00000008ecfe7800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-609220544, stack(0x00007fffe999f000,0x00007fffe9a9f000)]
>   0x00000008ecfe8800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-609218304, stack(0x00007fffe9aa0000,0x00007fffe9ba0000)]
>   0x00000008ecfe9000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-609216064, stack(0x00007fffe9ba1000,0x00007fffe9ca1000)]
>   0x00000008ecfea000 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-609213824, stack(0x00007fffe9ca2000,0x00007fffe9da2000)]
>   0x00000008ecfea800 JavaThread "_opt_soaps_store_6 indexers" daemon [_thread_blocked, id=-609211584, stack(0x00007fffe9da3000,0x00007fffe9ea3000)]
>   0x00000008e116e000 JavaThread "_opt_soaps_store_6v3 committer" daemon [_thread_blocked, id=-835879296, stack(0x00007fffedee4000,0x00007fffedfe4000)]
>   0x00000008e116f800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-517625920, stack(0x00007fffee5eb000,0x00007fffee6eb000)]
>   0x00000008e1191000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-669176128, stack(0x00007ffff181d000,0x00007ffff191d000)]
>   0x00000008e1249000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-576133312, stack(0x00007ffff161b000,0x00007ffff171b000)]
>   0x00000008e1495000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-683919936, stack(0x00007fffee3e9000,0x00007fffee4e9000)]
>   0x00000008e1495800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-571296192, stack(0x00007fffed2d8000,0x00007fffed3d8000)]
>   0x00000008e1496800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-542276736, stack(0x00007fffedfe5000,0x00007fffee0e5000)]
>   0x00000008e1497000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-546075456, stack(0x00007fffedce2000,0x00007fffedde2000)]
>   0x00000008e1498000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-397384320, stack(0x00007fffeeaf0000,0x00007fffeebf0000)]
>   0x00000008e1498800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-575029568, stack(0x00007ffffccd1000,0x00007ffffcdd1000)]
>   0x00000008e1499800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-576141824, stack(0x00007fffee7ed000,0x00007fffee8ed000)]
>   0x00000008e149a000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-546787712, stack(0x00007fffee6ec000,0x00007fffee7ec000)]
>   0x00000008e149b000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-483068160, stack(0x00007fffee1e7000,0x00007fffee2e7000)]
>   0x00000008e149b800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-483052928, stack(0x00007ffff6a6f000,0x00007ffff6b6f000)]
>   0x00000008e1a14800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-621599744, stack(0x00007fffee0e6000,0x00007fffee1e6000)]
>   0x00000008e1a15800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-549715904, stack(0x00007ffff151a000,0x00007ffff161a000)]
>   0x00000008e1a17000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-625499648, stack(0x00007ffffcbd0000,0x00007ffffccd0000)]
>   0x00000008e1a17800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=14434816, stack(0x00007fffed9df000,0x00007fffedadf000)]
>   0x00000008e1a18800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-581372992, stack(0x00007fffed1d7000,0x00007fffed2d7000)]
>   0x00000008e1a19000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-677618624, stack(0x00007fffecfd5000,0x00007fffed0d5000)]
>   0x00000008e1a1a000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-482648384, stack(0x00007fffecdd3000,0x00007fffeced3000)]
>   0x00000008e1a1a800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-483065920, stack(0x00007fffeced4000,0x00007fffecfd4000)]
>   0x00000008e1a1b800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-581385984, stack(0x00007fffed5db000,0x00007fffed6db000)]
>   0x00000008e2457000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-581370752, stack(0x00007fffed8de000,0x00007fffed9de000)]
>   0x00000008e2457800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-621396800, stack(0x00007ffff565b000,0x00007ffff575b000)]
>   0x00000008e2458800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-581381952, stack(0x00007fffed3d9000,0x00007fffed4d9000)]
>   0x00000008e2459000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-581379712, stack(0x00007ffff555a000,0x00007ffff565a000)]
>   0x00000008e245a000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-575034048, stack(0x00007fffed4da000,0x00007fffed5da000)]
>   0x00000008e245a800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-581390464, stack(0x00007fffedbe1000,0x00007fffedce1000)]
>   0x00000008e245b800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-581375232, stack(0x00007ffff666b000,0x00007ffff676b000)]
>   0x00000008e245c000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-483055168, stack(0x00007ffffcdd2000,0x00007ffffced2000)]
>   0x00000008e245d000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-483046208, stack(0x00007ffff5459000,0x00007ffff5559000)]
>   0x00000008e245d800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-581388224, stack(0x00007fffeccd2000,0x00007fffecdd2000)]
>   0x00000008e245e800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-483050688, stack(0x00007ffff575c000,0x00007ffff585c000)]
>   0x00000008e245f000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-581385536, stack(0x00007fffed0d6000,0x00007fffed1d6000)]
>   0x00000008e26dc800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-483048448, stack(0x00007ffff696e000,0x00007ffff6a6e000)]
>   0x00000008e26dd000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-581385088, stack(0x00007fffec5cb000,0x00007fffec6cb000)]
>   0x00000008e26de000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-581384192, stack(0x00007ffff1f24000,0x00007ffff2024000)]
>   0x00000008e26de800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-621394560, stack(0x00007fffec4ca000,0x00007fffec5ca000)]
>   0x00000008e26df800 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-483043968, stack(0x00007ffff5257000,0x00007ffff5357000)]
>   0x00000008e26e0000 JavaThread "_opt_soaps_store_6v3 indexers" daemon [_thread_blocked, id=-482655104, stack(0x00007ffffe7ec000,0x00007ffffe8ec000)]
>   0x00000008dd14b800 JavaThread "ActiveMQ BrokerService.worker.1" daemon [_thread_blocked, id=-542251200, stack(0x00007ffff4f54000,0x00007ffff5054000)]
>   0x00000008e6914800 JavaThread "Java2D Disposer" daemon [_thread_blocked, id=-527943808, stack(0x00007ffff5156000,0x00007ffff5256000)]
>   0x00000008ded42800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-556410560, stack(0x00007fffecad0000,0x00007fffecbd0000)]
>   0x00000008ded44000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-556410112, stack(0x00007fffecbd1000,0x00007fffeccd1000)]
>   0x00000008ded44800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-677619072, stack(0x00007ffffe6eb000,0x00007ffffe7eb000)]
>   0x00000008e343b800 JavaThread "_opt_soaps_store_4 committer" daemon [_thread_blocked, id=-480635840, stack(0x00007fffeebf1000,0x00007fffeecf1000)]
>   0x00000008e343d000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480633600, stack(0x00007fffeecf2000,0x00007fffeedf2000)]
>   0x00000008e343d800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480631360, stack(0x00007fffeedf3000,0x00007fffeeef3000)]
>   0x00000008e343e800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480629120, stack(0x00007fffeeef4000,0x00007fffeeff4000)]
>   0x00000008e343f000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481547584, stack(0x00007fffeeff5000,0x00007fffef0f5000)]
>   0x00000008e3440000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480889536, stack(0x00007fffef0f6000,0x00007fffef1f6000)]
>   0x00000008e3440800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480887296, stack(0x00007fffef1f7000,0x00007fffef2f7000)]
>   0x00000008e3441800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480885056, stack(0x00007fffef2f8000,0x00007fffef3f8000)]
>   0x00000008e3442000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480882816, stack(0x00007fffef3f9000,0x00007fffef4f9000)]
>   0x00000008e3443000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480880576, stack(0x00007fffef4fa000,0x00007fffef5fa000)]
>   0x00000008e3443800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480878336, stack(0x00007fffef5fb000,0x00007fffef6fb000)]
>   0x00000008e3444800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480876096, stack(0x00007fffef6fc000,0x00007fffef7fc000)]
>   0x00000008e3445000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480873856, stack(0x00007fffef7fd000,0x00007fffef8fd000)]
>   0x00000008e3446000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480871616, stack(0x00007fffef8fe000,0x00007fffef9fe000)]
>   0x00000008e3446800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480869376, stack(0x00007fffef9ff000,0x00007fffefaff000)]
>   0x00000008e3447800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480867136, stack(0x00007fffefb00000,0x00007fffefc00000)]
>   0x00000008e3449000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-480864896, stack(0x00007fffefc01000,0x00007fffefd01000)]
>   0x00000008e3449800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481574016, stack(0x00007fffefd02000,0x00007fffefe02000)]
>   0x00000008e344a800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481571776, stack(0x00007fffefe03000,0x00007fffeff03000)]
>   0x00000008e344b000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481569536, stack(0x00007fffeff04000,0x00007ffff0004000)]
>   0x00000008e344c000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481567296, stack(0x00007ffff0005000,0x00007ffff0105000)]
>   0x00000008e344c800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481565056, stack(0x00007ffff0106000,0x00007ffff0206000)]
>   0x00000008e344d800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481562816, stack(0x00007ffff0207000,0x00007ffff0307000)]
>   0x00000008e344e000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481560576, stack(0x00007ffff0308000,0x00007ffff0408000)]
>   0x00000008e344f000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481558336, stack(0x00007ffff0409000,0x00007ffff0509000)]
>   0x00000008e344f800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481556096, stack(0x00007ffff050a000,0x00007ffff060a000)]
>   0x00000008e3450800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481553856, stack(0x00007ffff060b000,0x00007ffff070b000)]
>   0x00000008e3451000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481551616, stack(0x00007ffff070c000,0x00007ffff080c000)]
>   0x00000008e3452000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481549376, stack(0x00007ffff080d000,0x00007ffff090d000)]
>   0x00000008e3452800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481951296, stack(0x00007ffff090e000,0x00007ffff0a0e000)]
>   0x00000008e3453800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481949056, stack(0x00007ffff0a0f000,0x00007ffff0b0f000)]
>   0x00000008e3455000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481946816, stack(0x00007ffff0b10000,0x00007ffff0c10000)]
>   0x00000008e3455800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481944576, stack(0x00007ffff0c11000,0x00007ffff0d11000)]
>   0x00000008e3456800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481942336, stack(0x00007ffff0d12000,0x00007ffff0e12000)]
>   0x00000008e3457000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481940096, stack(0x00007ffff0e13000,0x00007ffff0f13000)]
>   0x00000008e3458000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481937856, stack(0x00007ffff0f14000,0x00007ffff1014000)]
>   0x00000008e3458800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481935616, stack(0x00007ffff1015000,0x00007ffff1115000)]
>   0x00000008e3459800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481933376, stack(0x00007ffff1116000,0x00007ffff1216000)]
>   0x00000008e345a000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481931136, stack(0x00007ffff1217000,0x00007ffff1317000)]
>   0x00000008e345b000 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481928896, stack(0x00007ffff1318000,0x00007ffff1418000)]
>   0x00000008e345b800 JavaThread "_opt_soaps_store_4 indexers" daemon [_thread_blocked, id=-481926656, stack(0x00007ffff1419000,0x00007ffff1519000)]
>   0x00000008e1593800 JavaThread "http-nio-8090-exec-10" daemon [_thread_blocked, id=-623718336, stack(0x00007ffff6c71000,0x00007ffff6d71000)]
>   0x00000008e1594000 JavaThread "http-nio-8090-exec-9" daemon [_thread_blocked, id=-575031808, stack(0x00007ffffced3000,0x00007ffffcfd3000)]
>   0x00000008e1595000 JavaThread "http-nio-8090-exec-8" daemon [_thread_blocked, id=-623720576, stack(0x00007ffff6b70000,0x00007ffff6c70000)]
>   0x00000008e1595800 JavaThread "http-nio-8090-exec-7" daemon [_thread_blocked, id=-601531392, stack(0x00007ffff191e000,0x00007ffff1a1e000)]
>   0x00000008dcf95000 JavaThread "http-nio-8090-exec-6" daemon [_thread_blocked, id=-835878848, stack(0x00007ffff1a1f000,0x00007ffff1b1f000)]
>   0x00000008dcf95800 JavaThread "http-nio-8090-exec-5" daemon [_thread_blocked, id=-575025088, stack(0x00007ffffd1d6000,0x00007ffffd2d6000)]
>   0x00000008ded45800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-556409664, stack(0x00007ffff1c21000,0x00007ffff1d21000)]
>   0x00000008ded46000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-571295744, stack(0x00007ffff6368000,0x00007ffff6468000)]
>   0x00000008dcf96800 JavaThread "http-nio-8090-exec-4" daemon [_thread_blocked, id=-666135168, stack(0x00007ffffeaef000,0x00007ffffebef000)]
>   0x00000008dcf97000 JavaThread "http-nio-8090-exec-3" daemon [_thread_blocked, id=-694650048, stack(0x00007ffffebf0000,0x00007ffffecf0000)]
>   0x00000008dcf98000 JavaThread "http-nio-8090-exec-2" daemon [_thread_blocked, id=-576157952, stack(0x00007ffffd3d8000,0x00007ffffd4d8000)]
>   0x00000008ded47000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-556407424, stack(0x00007ffff1d22000,0x00007ffff1e22000)]
>   0x00000008ded47800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-556406080, stack(0x00007ffff1e23000,0x00007ffff1f23000)]
>   0x00000008dff70800 JavaThread "audit committer" daemon [_thread_blocked, id=-544194752, stack(0x00007ffff494e000,0x00007ffff4a4e000)]
>   0x00000008dff71000 JavaThread "audit indexers" daemon [_thread_blocked, id=-544192512, stack(0x00007ffff4a4f000,0x00007ffff4b4f000)]
>   0x00000008dff72000 JavaThread "audit indexers" daemon [_thread_blocked, id=-544190272, stack(0x00007ffff4b50000,0x00007ffff4c50000)]
>   0x00000008dff72800 JavaThread "audit indexers" daemon [_thread_blocked, id=-544188032, stack(0x00007ffff4c51000,0x00007ffff4d51000)]
>   0x00000008dff73800 JavaThread "audit indexers" daemon [_thread_blocked, id=-544185792, stack(0x00007ffff4d52000,0x00007ffff4e52000)]
>   0x00000008db249000 JavaThread "pxmessagestream" daemon [_thread_blocked, id=-621597504, stack(0x00007ffff5b60000,0x00007ffff5c60000)]
>   0x00000008e0f76800 JavaThread "pxmessagestream" daemon [_thread_blocked, id=-521688704, stack(0x00007ffff5c61000,0x00007ffff5d61000)]
>   0x00000008e0dd5800 JavaThread "pxmessagestream" daemon [_thread_blocked, id=-522294464, stack(0x00007ffff5d62000,0x00007ffff5e62000)]
>   0x00000008dfce5000 JavaThread "pxmessagestream" daemon [_thread_blocked, id=-540106304, stack(0x00007ffff5e63000,0x00007ffff5f63000)]
>   0x00000008e0cd9000 JavaThread "pxmessagestream" daemon [_thread_blocked, id=-523362624, stack(0x00007ffff5f64000,0x00007ffff6064000)]
>   0x00000008e0c6c800 JavaThread "pxmessagestream" daemon [_thread_blocked, id=-523710336, stack(0x00007ffff6065000,0x00007ffff6165000)]
>   0x00000008e0c3b000 JavaThread "pxmessagestream" daemon [_thread_blocked, id=-524307904, stack(0x00007ffff6166000,0x00007ffff6266000)]
>   0x00000008db531800 JavaThread "pxmessagestream" daemon [_thread_blocked, id=-620773760, stack(0x00007ffff6267000,0x00007ffff6367000)]
>   0x00000008dcf98800 JavaThread "http-nio-8090-exec-1" daemon [_thread_blocked, id=-587570688, stack(0x00007ffff6e73000,0x00007ffff6f73000)]
>   0x0000000801efc800 JavaThread "ajp-apr-8009-AsyncTimeout" daemon [_thread_blocked, id=-694667968, stack(0x00007ffff6f74000,0x00007ffff7074000)]
>   0x0000000801efe000 JavaThread "ajp-apr-8009-Acceptor-0" daemon [_thread_in_native, id=-694665728, stack(0x00007ffff7075000,0x00007ffff7175000)]
>   0x0000000801efe800 JavaThread "ajp-apr-8009-CometPoller-0" daemon [_thread_blocked, id=-694663488, stack(0x00007ffff7176000,0x00007ffff7276000)]
>   0x0000000801eff800 JavaThread "ajp-apr-8009-Poller-0" daemon [_thread_blocked, id=-694661248, stack(0x00007ffff7277000,0x00007ffff7377000)]
>   0x0000000801f00000 JavaThread "http-nio-8090-Acceptor-0" daemon [_thread_in_native, id=-694659008, stack(0x00007ffff7378000,0x00007ffff7478000)]
>   0x0000000801f01000 JavaThread "http-nio-8090-ClientPoller-1" daemon [_thread_in_native, id=-694656768, stack(0x00007ffff7479000,0x00007ffff7579000)]
>   0x0000000801f01800 JavaThread "http-nio-8090-ClientPoller-0" daemon [_thread_in_native, id=-694654528, stack(0x00007ffff757a000,0x00007ffff767a000)]
>   0x0000000801f02800 JavaThread "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon [_thread_blocked, id=-694652288, stack(0x00007ffff767b000,0x00007ffff777b000)]
>   0x00000008d7ba7800 JavaThread "Comet[]" daemon [_thread_blocked, id=-681627776, stack(0x00007ffff777c000,0x00007ffff787c000)]
>   0x00000008dd38b000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-571293504, stack(0x00007ffff787d000,0x00007ffff797d000)]
>   0x00000008dd38b800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-571291264, stack(0x00007ffff797e000,0x00007ffff7a7e000)]
>   0x00000008dd38c800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-571289024, stack(0x00007ffff7a7f000,0x00007ffff7b7f000)]
>   0x00000008dd38d000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-571286784, stack(0x00007ffff7b80000,0x00007ffff7c80000)]
>   0x00000008dd38e000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-571284544, stack(0x00007ffff7c81000,0x00007ffff7d81000)]
>   0x00000008dd38e800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-571282304, stack(0x00007ffff7d82000,0x00007ffff7e82000)]
>   0x00000008dd38f800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-571250944, stack(0x00007ffff7e83000,0x00007ffff7f83000)]
>   0x00000008dd391000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-583589056, stack(0x00007ffff7f84000,0x00007ffff8084000)]
>   0x00000008dd391800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-583586816, stack(0x00007ffff8085000,0x00007ffff8185000)]
>   0x00000008dd392800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-583584576, stack(0x00007ffff8186000,0x00007ffff8286000)]
>   0x00000008dd393000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-583582336, stack(0x00007ffff8287000,0x00007ffff8387000)]
>   0x00000008dd394000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-583580096, stack(0x00007ffff8388000,0x00007ffff8488000)]
>   0x00000008dd394800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-583577856, stack(0x00007ffff8489000,0x00007ffff8589000)]
>   0x00000008dd395800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-583575616, stack(0x00007ffff858a000,0x00007ffff868a000)]
>   0x00000008dd396000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-583573376, stack(0x00007ffff868b000,0x00007ffff878b000)]
>   0x00000008dd397000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-583571136, stack(0x00007ffff878c000,0x00007ffff888c000)]
>   0x00000008dd397800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-583568896, stack(0x00007ffff888d000,0x00007ffff898d000)]
>   0x00000008dd398800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-583566656, stack(0x00007ffff898e000,0x00007ffff8a8e000)]
>   0x00000008dd399000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-571308736, stack(0x00007ffff8a8f000,0x00007ffff8b8f000)]
>   0x00000008dd39a000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-571306496, stack(0x00007ffff8b90000,0x00007ffff8c90000)]
>   0x00000008dd39a800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-584979904, stack(0x00007ffff8c91000,0x00007ffff8d91000)]
>   0x00000008dd39b800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-584977664, stack(0x00007ffff8d92000,0x00007ffff8e92000)]
>   0x00000008dd39d000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-584975424, stack(0x00007ffff8e93000,0x00007ffff8f93000)]
>   0x00000008dd39d800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-584973184, stack(0x00007ffff8f94000,0x00007ffff9094000)]
>   0x00000008dd39e800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-584970944, stack(0x00007ffff9095000,0x00007ffff9195000)]
>   0x00000008dd39f000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-584968704, stack(0x00007ffff9196000,0x00007ffff9296000)]
>   0x00000008dd3a0000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-584966464, stack(0x00007ffff9297000,0x00007ffff9397000)]
>   0x00000008dd3a0800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-584964224, stack(0x00007ffff9398000,0x00007ffff9498000)]
>   0x00000008dd3a1800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-584961984, stack(0x00007ffff9499000,0x00007ffff9599000)]
>   0x00000008dd3a2000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-584959744, stack(0x00007ffff959a000,0x00007ffff969a000)]
>   0x00000008dd3a3000 JavaThread "mailbox worker" daemon [_thread_blocked, id=-681617024, stack(0x00007ffff999e000,0x00007ffff9a9e000)]
>   0x00000008dd3a3800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-576155712, stack(0x00007ffffd4d9000,0x00007ffffd5d9000)]
>   0x00000008dd3a4800 JavaThread "mailbox worker" daemon [_thread_blocked, id=-677593984, stack(0x00007ffffd2d7000,0x00007ffffd3d7000)]
>   0x00000008d7ba8800 JavaThread "imap/pop:192.168.1.10" daemon [_thread_in_native, id=-576146304, stack(0x00007ffffdde2000,0x00007ffffdee2000)]
>   0x00000008d7ba9000 JavaThread "lcheck" daemon [_thread_blocked, id=-694641088, stack(0x00007ffffe4e9000,0x00007ffffe5e9000)]
>   0x00000008d7baa000 JavaThread "log" daemon [_thread_blocked, id=-681623744, stack(0x00007ffff969b000,0x00007ffff979b000)]
>   0x00000008d7baa800 JavaThread "pdfbox" daemon [_thread_blocked, id=-681621504, stack(0x00007ffff979c000,0x00007ffff989c000)]
>   0x00000008d7bab800 JavaThread "license" daemon [_thread_blocked, id=-681619264, stack(0x00007ffff989d000,0x00007ffff999d000)]
>   0x00000008d7562800 JavaThread "auto update" daemon [_thread_blocked, id=-681614784, stack(0x00007ffff9a9f000,0x00007ffff9b9f000)]
>   0x00000008d7563800 JavaThread "stubbing" daemon [_thread_blocked, id=-681612544, stack(0x00007ffff9ba0000,0x00007ffff9ca0000)]
>   0x00000008d7564000 JavaThread "purge" daemon [_thread_blocked, id=-688138432, stack(0x00007ffff9ca1000,0x00007ffff9da1000)]
>   0x00000008d7565000 JavaThread "pool-51-thread-1" [_thread_blocked, id=-682012224, stack(0x00007ffff9da2000,0x00007ffff9ea2000)]
>   0x00000008d7565800 JavaThread "diskspace" daemon [_thread_blocked, id=-682009984, stack(0x00007ffff9ea3000,0x00007ffff9fa3000)]
>   0x00000008d7566800 JavaThread "config auto load" daemon [_thread_blocked, id=-682009536, stack(0x00007ffff9fa4000,0x00007ffffa0a4000)]
>   0x00000008d7567000 JavaThread "volume status detection" daemon [_thread_blocked, id=-682007296, stack(0x00007ffffa0a5000,0x00007ffffa1a5000)]
>   0x00000008d7568000 JavaThread "receive" daemon [_thread_blocked, id=-682005056, stack(0x00007ffffa1a6000,0x00007ffffa2a6000)]
>   0x00000008d7568800 JavaThread "receive" daemon [_thread_blocked, id=-682002816, stack(0x00007ffffa2a7000,0x00007ffffa3a7000)]
>   0x00000008d7569800 JavaThread "receive" daemon [_thread_blocked, id=-682000576, stack(0x00007ffffa3a8000,0x00007ffffa4a8000)]
>   0x00000008d756a000 JavaThread "receive" daemon [_thread_blocked, id=-681998336, stack(0x00007ffffa4a9000,0x00007ffffa5a9000)]
>   0x00000008d756b000 JavaThread "receive" daemon [_thread_blocked, id=-681996096, stack(0x00007ffffa5aa000,0x00007ffffa6aa000)]
>   0x00000008d756b800 JavaThread "receive" daemon [_thread_blocked, id=-681993856, stack(0x00007ffffa6ab000,0x00007ffffa7ab000)]
>   0x00000008d756c800 JavaThread "receive" daemon [_thread_blocked, id=-681991616, stack(0x00007ffffa7ac000,0x00007ffffa8ac000)]
>   0x00000008d7578000 JavaThread "receive" daemon [_thread_blocked, id=-681989376, stack(0x00007ffffa8ad000,0x00007ffffa9ad000)]
>   0x00000008d7578800 JavaThread "receive" daemon [_thread_blocked, id=-694610112, stack(0x00007ffffa9ae000,0x00007ffffaaae000)]
>   0x00000008d7579800 JavaThread "receive" daemon [_thread_blocked, id=-688164416, stack(0x00007ffffaaaf000,0x00007ffffabaf000)]
>   0x00000008d757a000 JavaThread "receive" daemon [_thread_blocked, id=-688162176, stack(0x00007ffffabb0000,0x00007ffffacb0000)]
>   0x00000008d757b000 JavaThread "receive" daemon [_thread_blocked, id=-688159936, stack(0x00007ffffacb1000,0x00007ffffadb1000)]
>   0x00000008d757b800 JavaThread "receive" daemon [_thread_blocked, id=-688157696, stack(0x00007ffffadb2000,0x00007ffffaeb2000)]
>   0x00000008d757c800 JavaThread "receive" daemon [_thread_blocked, id=-688157248, stack(0x00007ffffaeb3000,0x00007ffffafb3000)]
>   0x00000008d757d000 JavaThread "receive" daemon [_thread_blocked, id=-688155008, stack(0x00007ffffafb4000,0x00007ffffb0b4000)]
>   0x00000008d757e000 JavaThread "receive" daemon [_thread_blocked, id=-688152768, stack(0x00007ffffb0b5000,0x00007ffffb1b5000)]
>   0x00000008d757e800 JavaThread "receive" daemon [_thread_blocked, id=-688150528, stack(0x00007ffffb1b6000,0x00007ffffb2b6000)]
>   0x00000008d757f800 JavaThread "receive" daemon [_thread_blocked, id=-688148288, stack(0x00007ffffb2b7000,0x00007ffffb3b7000)]
>   0x00000008d7580000 JavaThread "receive" daemon [_thread_blocked, id=-688146048, stack(0x00007ffffb3b8000,0x00007ffffb4b8000)]
>   0x00000008d7581000 JavaThread "receive" daemon [_thread_blocked, id=-688143808, stack(0x00007ffffb4b9000,0x00007ffffb5b9000)]
>   0x00000008d7581800 JavaThread "receive" daemon [_thread_blocked, id=-688141568, stack(0x00007ffffb5ba000,0x00007ffffb6ba000)]
>   0x00000008d7582800 JavaThread "receive" daemon [_thread_blocked, id=-575023296, stack(0x00007ffffb6bb000,0x00007ffffb7bb000)]
>   0x00000008d7d66000 JavaThread "receive" daemon [_thread_blocked, id=-694636096, stack(0x00007ffffb7bc000,0x00007ffffb8bc000)]
>   0x00000008d7d66800 JavaThread "receive" daemon [_thread_blocked, id=-694633856, stack(0x00007ffffb8bd000,0x00007ffffb9bd000)]
>   0x00000008d7d67800 JavaThread "receive" daemon [_thread_blocked, id=-694631616, stack(0x00007ffffb9be000,0x00007ffffbabe000)]
>   0x00000008d7d68000 JavaThread "receive" daemon [_thread_blocked, id=-694629376, stack(0x00007ffffbabf000,0x00007ffffbbbf000)]
>   0x0000000801f0b800 JavaThread "receive" daemon [_thread_blocked, id=-694627136, stack(0x00007ffffbbc0000,0x00007ffffbcc0000)]
>   0x0000000801f0c000 JavaThread "receive" daemon [_thread_blocked, id=-694624896, stack(0x00007ffffbcc1000,0x00007ffffbdc1000)]
>   0x0000000801f0d000 JavaThread "receive" daemon [_thread_blocked, id=-694622656, stack(0x00007ffffbdc2000,0x00007ffffbec2000)]
>   0x0000000801f0d800 JavaThread "receive" daemon [_thread_blocked, id=-694620416, stack(0x00007ffffbec3000,0x00007ffffbfc3000)]
>   0x0000000801f0e800 JavaThread "receive" daemon [_thread_blocked, id=-694618176, stack(0x00007ffffbfc4000,0x00007ffffc0c4000)]
>   0x0000000801f0f000 JavaThread "receive" daemon [_thread_blocked, id=-694615936, stack(0x00007ffffc0c5000,0x00007ffffc1c5000)]
>   0x0000000801f10000 JavaThread "receive" daemon [_thread_blocked, id=-694613696, stack(0x00007ffffc1c6000,0x00007ffffc2c6000)]
>   0x0000000801f10800 JavaThread "receive" daemon [_thread_blocked, id=-694611456, stack(0x00007ffffc2c7000,0x00007ffffc3c7000)]
>   0x0000000801f11800 JavaThread "receive" daemon [_thread_blocked, id=-576131520, stack(0x00007ffffc3c8000,0x00007ffffc4c8000)]
>   0x0000000801f12000 JavaThread "receive" daemon [_thread_blocked, id=-575047488, stack(0x00007ffffc4c9000,0x00007ffffc5c9000)]
>   0x0000000801f13000 JavaThread "receive" daemon [_thread_blocked, id=-575045248, stack(0x00007ffffc5ca000,0x00007ffffc6ca000)]
>   0x0000000801f13800 JavaThread "receive" daemon [_thread_blocked, id=-575043008, stack(0x00007ffffc6cb000,0x00007ffffc7cb000)]
>   0x0000000801f14800 JavaThread "receive" daemon [_thread_blocked, id=-575040768, stack(0x00007ffffc7cc000,0x00007ffffc8cc000)]
>   0x00000008d7b7d000 JavaThread "receive" daemon [_thread_blocked, id=-575038528, stack(0x00007ffffc8cd000,0x00007ffffc9cd000)]
>   0x00000008d7b7d800 JavaThread "receive" daemon [_thread_blocked, id=-575036288, stack(0x00007ffffc9ce000,0x00007ffffcace000)]
>   0x00000008d7b80800 JavaThread "diskspacechecker" daemon [_thread_blocked, id=-575027328, stack(0x00007ffffd0d5000,0x00007ffffd1d5000)]
>   0x00000008d7b84800 JavaThread "statusreport" daemon [_thread_blocked, id=-576153472, stack(0x00007ffffd5da000,0x00007ffffd6da000)]
>   0x00000008d7b85000 JavaThread "statusreport" daemon [_thread_blocked, id=-576151232, stack(0x00007ffffd6db000,0x00007ffffd7db000)]
>   0x00000008d8523000 JavaThread "RMI RenewClean-[192.168.1.2:45755]" daemon [_thread_in_native, id=-666066432, stack(0x00007ffffd8dd000,0x00007ffffd9dd000)]
>   0x0000000800c94800 JavaThread "RMI Scheduler(0)" daemon [_thread_blocked, id=13264768, stack(0x00007ffffd9de000,0x00007ffffdade000)]
>   0x0000000800c96000 JavaThread "RMI Reaper" [_thread_blocked, id=13266560, stack(0x00007ffffdbe0000,0x00007ffffdce0000)]
>   0x0000000800c96800 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=-576146752, stack(0x00007ffffdce1000,0x00007ffffdde1000)]
>   0x00000008d7b86800 JavaThread "RMI TCP Accept-1099" daemon [_thread_in_native, id=-576141376, stack(0x00007ffffe0e5000,0x00007ffffe1e5000)]
>   0x00000008d7b87800 JavaThread "ActiveMQ Broker[ROOT] Scheduler" daemon [_thread_blocked, id=-576140928, stack(0x00007ffffe1e6000,0x00007ffffe2e6000)]
>   0x00000008d7b9a000 JavaThread "deleteHelper" daemon [_thread_blocked, id=-677619520, stack(0x00007ffffe5ea000,0x00007ffffe6ea000)]
>   0x00000008d7b9e000 JavaThread "AsyncAppender-Worker-Thread-4" daemon [_thread_blocked, id=-677606528, stack(0x00007ffffe8ed000,0x00007ffffe9ed000)]
>   0x00000008d7b9d000 JavaThread "AsyncAppender-Worker-Thread-3" daemon [_thread_blocked, id=-677604288, stack(0x00007ffffe9ee000,0x00007ffffeaee000)]
>   0x0000000801f04000 JavaThread "NioBlockingSelector.BlockPoller-1" daemon [_thread_in_native, id=-694647808, stack(0x00007ffffecf1000,0x00007ffffedf1000)]
>   0x0000000801f04800 JavaThread "GC Daemon" daemon [_thread_blocked, id=-694645568, stack(0x00007ffffedf2000,0x00007ffffeef2000)]
>   0x0000000801f05800 JavaThread "Service Thread" daemon [_thread_blocked, id=31900544, stack(0x00007ffffeff4000,0x00007fffff0f4000)]
>   0x0000000801f06000 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=31902784, stack(0x00007fffff0f5000,0x00007fffff1f5000)]
>   0x0000000801f07000 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=31905024, stack(0x00007fffff1f6000,0x00007fffff2f6000)]
>   0x0000000801f07800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=31907264, stack(0x00007fffff2f7000,0x00007fffff3f7000)]
>   0x0000000801f08800 JavaThread "Finalizer" daemon [_thread_blocked, id=31909504, stack(0x00007fffff3f8000,0x00007fffff4f8000)]
>   0x0000000801f0a000 JavaThread "Reference Handler" daemon [_thread_blocked, id=31911744, stack(0x00007fffff4f9000,0x00007fffff5f9000)]
>   0x0000000801f0a800 JavaThread "main" [_thread_in_native, id=12627520, stack(0x00007fffffaff000,0x00007fffffbff000)]
> 
> Other Threads:
>   0x0000000801f48800 VMThread [stack: 0x00007fffff5fa000,0x00007fffff6fa000] [id=31913984]
>   0x0000000801f48000 WatcherThread [stack: 0x00007ffffeef3000,0x00007ffffeff3000] [id=-694642880]
> 
> VM state:not at safepoint (normal execution)
> 
> VM Mutex/Monitor currently owned by a thread: None
> 
> Heap
>  PSYoungGen      total 1040576K, used 398287K [0x000000088d600000, 0x00000008cd750000, 0x00000008cd750000)
>   eden space 1032384K, 38% used [0x000000088d600000,0x00000008a552f458,0x00000008cc630000)
>   from space 8192K, 72% used [0x00000008cc630000,0x00000008ccbf49b8,0x00000008cce30000)
>   to   space 8384K, 0% used [0x00000008ccf20000,0x00000008ccf20000,0x00000008cd750000)
>  ParOldGen       total 2099904K, used 398032K [0x000000080d350000, 0x000000088d600000, 0x000000088d600000)
>   object space 2099904K, 18% used [0x000000080d350000,0x0000000825804178,0x000000088d600000)
>  PSPermGen       total 131072K, used 93251K [0x0000000805350000, 0x000000080d350000, 0x000000080d350000)
>   object space 131072K, 71% used [0x0000000805350000,0x000000080ae60f48,0x000000080d350000)
> 
> Card table byte_map: [0x00000008cd750000,0x00000008cdd93000] byte_map_base: 0x00000008c9726580
> 
> Polling page: 0x000000080054c000
> 
> Code Cache  [0x000000080233d000, 0x000000080436d000, 0x000000080533d000)
>  total_blobs=9033 nmethods=8441 adapters=542 free_code_cache=16527Kb largest_free_block=16643712
> 
> Compilation events (10 events):
> Event: 769044.059 Thread 0x0000000801f07000 9396   !         java.security.cert.Certificate::hashCode (34 bytes)
> Event: 769044.064 Thread 0x0000000801f07000 nmethod 9396 0x0000000804314310 code [0x0000000804314460, 0x0000000804314688]
> Event: 795408.913 Thread 0x0000000801f06000 9397   !         org.apache.lucene.index.DocumentsWriter::flushAllThreads (337 bytes)
> Event: 795408.959 Thread 0x0000000801f06000 nmethod 9397 0x000000080433bc10 code [0x000000080433be00, 0x000000080433c400]
> Event: 802631.467 Thread 0x0000000801f07000 9398             org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy::onCommit (36 bytes)
> Event: 802631.472 Thread 0x0000000801f07000 nmethod 9398 0x0000000804313c10 code [0x0000000804313da0, 0x0000000804314008]
> Event: 819768.213 Thread 0x0000000801f06000 9399   !         sun.reflect.GeneratedConstructorAccessor705::newInstance (53 bytes)
> Event: 819768.217 Thread 0x0000000801f06000 nmethod 9399 0x000000080433b710 code [0x000000080433b880, 0x000000080433ba28]
> Event: 819768.270 Thread 0x0000000801f07000 9400   !         sun.reflect.GeneratedMethodAccessor434::invoke (66 bytes)
> Event: 819768.272 Thread 0x0000000801f07000 nmethod 9400 0x000000080433b2d0 code [0x000000080433b420, 0x000000080433b578]
> 
> GC Heap History (10 events):
> Event: 849003.532 GC heap before
> {Heap before GC invocations=49522 (full 383):
>  PSYoungGen      total 1041600K, used 1037184K [0x000000088d600000, 0x00000008cd680000, 0x00000008cd750000)
>   eden space 1033984K, 100% used [0x000000088d600000,0x00000008cc7c0000,0x00000008cc7c0000)
>   from space 7616K, 42% used [0x00000008ccf10000,0x00000008cd230020,0x00000008cd680000)
>   to   space 7488K, 0% used [0x00000008cc7c0000,0x00000008cc7c0000,0x00000008ccf10000)
>  ParOldGen       total 2099904K, used 395548K [0x000000080d350000, 0x000000088d600000, 0x000000088d600000)
>   object space 2099904K, 18% used [0x000000080d350000,0x0000000825597238,0x000000088d600000)
>  PSPermGen       total 131072K, used 93251K [0x0000000805350000, 0x000000080d350000, 0x000000080d350000)
>   object space 131072K, 71% used [0x0000000805350000,0x000000080ae60f48,0x000000080d350000)
> Event: 849003.547 GC heap after
> Heap after GC invocations=49522 (full 383):
>  PSYoungGen      total 1041472K, used 7487K [0x000000088d600000, 0x00000008cd750000, 0x00000008cd750000)
>   eden space 1033984K, 0% used [0x000000088d600000,0x000000088d600000,0x00000008cc7c0000)
>   from space 7488K, 99% used [0x00000008cc7c0000,0x00000008ccf0ffc8,0x00000008ccf10000)
>   to   space 8448K, 0% used [0x00000008ccf10000,0x00000008ccf10000,0x00000008cd750000)
>  ParOldGen       total 2099904K, used 397896K [0x000000080d350000, 0x000000088d600000, 0x000000088d600000)
>   object space 2099904K, 18% used [0x000000080d350000,0x00000008257e2178,0x000000088d600000)
>  PSPermGen       total 131072K, used 93251K [0x0000000805350000, 0x000000080d350000, 0x000000080d350000)
>   object space 131072K, 71% used [0x0000000805350000,0x000000080ae60f48,0x000000080d350000)
> }
> Event: 849030.522 GC heap before
> {Heap before GC invocations=49523 (full 383):
>  PSYoungGen      total 1041472K, used 1041471K [0x000000088d600000, 0x00000008cd750000, 0x00000008cd750000)
>   eden space 1033984K, 100% used [0x000000088d600000,0x00000008cc7c0000,0x00000008cc7c0000)
>   from space 7488K, 99% used [0x00000008cc7c0000,0x00000008ccf0ffc8,0x00000008ccf10000)
>   to   space 8448K, 0% used [0x00000008ccf10000,0x00000008ccf10000,0x00000008cd750000)
>  ParOldGen       total 2099904K, used 397896K [0x000000080d350000, 0x000000088d600000, 0x000000088d600000)
>   object space 2099904K, 18% used [0x000000080d350000,0x00000008257e2178,0x000000088d600000)
>  PSPermGen       total 131072K, used 93251K [0x0000000805350000, 0x000000080d350000, 0x000000080d350000)
>   object space 131072K, 71% used [0x0000000805350000,0x000000080ae60f48,0x000000080d350000)
> Event: 849030.535 GC heap after
> Heap after GC invocations=49523 (full 383):
>  PSYoungGen      total 1040576K, used 4511K [0x000000088d600000, 0x00000008cd750000, 0x00000008cd750000)
>   eden space 1032128K, 0% used [0x000000088d600000,0x000000088d600000,0x00000008cc5f0000)
>   from space 8448K, 53% used [0x00000008ccf10000,0x00000008cd377e98,0x00000008cd750000)
>   to   space 8896K, 0% used [0x00000008cc5f0000,0x00000008cc5f0000,0x00000008ccea0000)
>  ParOldGen       total 2099904K, used 397960K [0x000000080d350000, 0x000000088d600000, 0x000000088d600000)
>   object space 2099904K, 18% used [0x000000080d350000,0x00000008257f2178,0x000000088d600000)
>  PSPermGen       total 131072K, used 93251K [0x0000000805350000, 0x000000080d350000, 0x000000080d350000)
>   object space 131072K, 71% used [0x0000000805350000,0x000000080ae60f48,0x000000080d350000)
> }
> Event: 849057.512 GC heap before
> {Heap before GC invocations=49524 (full 383):
>  PSYoungGen      total 1040576K, used 1036639K [0x000000088d600000, 0x00000008cd750000, 0x00000008cd750000)
>   eden space 1032128K, 100% used [0x000000088d600000,0x00000008cc5f0000,0x00000008cc5f0000)
>   from space 8448K, 53% used [0x00000008ccf10000,0x00000008cd377e98,0x00000008cd750000)
>   to   space 8896K, 0% used [0x00000008cc5f0000,0x00000008cc5f0000,0x00000008ccea0000)
>  ParOldGen       total 2099904K, used 397960K [0x000000080d350000, 0x000000088d600000, 0x000000088d600000)
>   object space 2099904K, 18% used [0x000000080d350000,0x00000008257f2178,0x000000088d600000)
>  PSPermGen       total 131072K, used 93251K [0x0000000805350000, 0x000000080d350000, 0x000000080d350000)
>   object space 131072K, 71% used [0x0000000805350000,0x000000080ae60f48,0x000000080d350000)
> Event: 849057.525 GC heap after
> Heap after GC invocations=49524 (full 383):
>  PSYoungGen      total 1041024K, used 3360K [0x000000088d600000, 0x00000008cd740000, 0x00000008cd750000)
>   eden space 1032128K, 0% used [0x000000088d600000,0x000000088d600000,0x00000008cc5f0000)
>   from space 8896K, 37% used [0x00000008cc5f0000,0x00000008cc9383a8,0x00000008ccea0000)
>   to   space 8640K, 0% used [0x00000008cced0000,0x00000008cced0000,0x00000008cd740000)
>  ParOldGen       total 2099904K, used 397960K [0x000000080d350000, 0x000000088d600000, 0x000000088d600000)
>   object space 2099904K, 18% used [0x000000080d350000,0x00000008257f2178,0x000000088d600000)
>  PSPermGen       total 131072K, used 93251K [0x0000000805350000, 0x000000080d350000, 0x000000080d350000)
>   object space 131072K, 71% used [0x0000000805350000,0x000000080ae60f48,0x000000080d350000)
> }
> Event: 849084.515 GC heap before
> {Heap before GC invocations=49525 (full 383):
>  PSYoungGen      total 1041024K, used 1035488K [0x000000088d600000, 0x00000008cd740000, 0x00000008cd750000)
>   eden space 1032128K, 100% used [0x000000088d600000,0x00000008cc5f0000,0x00000008cc5f0000)
>   from space 8896K, 37% used [0x00000008cc5f0000,0x00000008cc9383a8,0x00000008ccea0000)
>   to   space 8640K, 0% used [0x00000008cced0000,0x00000008cced0000,0x00000008cd740000)
>  ParOldGen       total 2099904K, used 397960K [0x000000080d350000, 0x000000088d600000, 0x000000088d600000)
>   object space 2099904K, 18% used [0x000000080d350000,0x00000008257f2178,0x000000088d600000)
>  PSPermGen       total 131072K, used 93251K [0x0000000805350000, 0x000000080d350000, 0x000000080d350000)
>   object space 131072K, 71% used [0x0000000805350000,0x000000080ae60f48,0x000000080d350000)
> Event: 849084.527 GC heap after
> Heap after GC invocations=49525 (full 383):
>  PSYoungGen      total 1039936K, used 4254K [0x000000088d600000, 0x00000008cd630000, 0x00000008cd750000)
>   eden space 1032384K, 0% used [0x000000088d600000,0x000000088d600000,0x00000008cc630000)
>   from space 7552K, 56% used [0x00000008cced0000,0x00000008cd2f7918,0x00000008cd630000)
>   to   space 8192K, 0% used [0x00000008cc630000,0x00000008cc630000,0x00000008cce30000)
>  ParOldGen       total 2099904K, used 397992K [0x000000080d350000, 0x000000088d600000, 0x000000088d600000)
>   object space 2099904K, 18% used [0x000000080d350000,0x00000008257fa178,0x000000088d600000)
>  PSPermGen       total 131072K, used 93251K [0x0000000805350000, 0x000000080d350000, 0x000000080d350000)
>   object space 131072K, 71% used [0x0000000805350000,0x000000080ae60f48,0x000000080d350000)
> }
> Event: 849111.980 GC heap before
> {Heap before GC invocations=49526 (full 383):
>  PSYoungGen      total 1039936K, used 1036638K [0x000000088d600000, 0x00000008cd630000, 0x00000008cd750000)
>   eden space 1032384K, 100% used [0x000000088d600000,0x00000008cc630000,0x00000008cc630000)
>   from space 7552K, 56% used [0x00000008cced0000,0x00000008cd2f7918,0x00000008cd630000)
>   to   space 8192K, 0% used [0x00000008cc630000,0x00000008cc630000,0x00000008cce30000)
>  ParOldGen       total 2099904K, used 397992K [0x000000080d350000, 0x000000088d600000, 0x000000088d600000)
>   object space 2099904K, 18% used [0x000000080d350000,0x00000008257fa178,0x000000088d600000)
>  PSPermGen       total 131072K, used 93251K [0x0000000805350000, 0x000000080d350000, 0x000000080d350000)
>   object space 131072K, 71% used [0x0000000805350000,0x000000080ae60f48,0x000000080d350000)
> Event: 849111.994 GC heap after
> Heap after GC invocations=49526 (full 383):
>  PSYoungGen      total 1040576K, used 5906K [0x000000088d600000, 0x00000008cd750000, 0x00000008cd750000)
>   eden space 1032384K, 0% used [0x000000088d600000,0x000000088d600000,0x00000008cc630000)
>   from space 8192K, 72% used [0x00000008cc630000,0x00000008ccbf49b8,0x00000008cce30000)
>   to   space 8384K, 0% used [0x00000008ccf20000,0x00000008ccf20000,0x00000008cd750000)
>  ParOldGen       total 2099904K, used 398032K [0x000000080d350000, 0x000000088d600000, 0x000000088d600000)
>   object space 2099904K, 18% used [0x000000080d350000,0x0000000825804178,0x000000088d600000)
>  PSPermGen       total 131072K, used 93251K [0x0000000805350000, 0x000000080d350000, 0x000000080d350000)
>   object space 131072K, 71% used [0x0000000805350000,0x000000080ae60f48,0x000000080d350000)
> }
> 
> Deoptimization events (10 events):
> Event: 688223.473 Thread 0x00000008d7ba7800 Uncommon trap -10 fr.pc 0x0000000802fafe98
> Event: 688223.565 Thread 0x00000008e1595000 Uncommon trap -58 fr.pc 0x0000000802a638c4
> Event: 688371.497 Thread 0x00000008e1594000 Uncommon trap -58 fr.pc 0x00000008035ae974
> Event: 688371.500 Thread 0x00000008e1594000 Uncommon trap -58 fr.pc 0x0000000803f29324
> Event: 688371.500 Thread 0x00000008e1594000 Uncommon trap -34 fr.pc 0x00000008024cc304
> Event: 688371.501 Thread 0x00000008e1594000 Uncommon trap -34 fr.pc 0x00000008024080b4
> Event: 688371.501 Thread 0x00000008e1594000 Uncommon trap -58 fr.pc 0x0000000803996370
> Event: 688371.501 Thread 0x00000008e1594000 Uncommon trap -58 fr.pc 0x0000000803fd0b28
> Event: 689571.393 Thread 0x00000008d7b9a000 Uncommon trap -34 fr.pc 0x0000000803b926f8
> Event: 689571.393 Thread 0x00000008d7b9a000 Uncommon trap -34 fr.pc 0x00000008036caf0c
> 
> Internal exceptions (10 events):
> Event: 849026.107 Thread 0x00000008d7562800 Threw 0x00000008c23e7a20 at /opt/soaps/store/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:716
> Event: 849026.304 Thread 0x00000008d7562800 Threw 0x00000008c241a148 at /opt/soaps/store/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:716
> Event: 849026.494 Thread 0x00000008d7562800 Threw 0x00000008c241d0a8 at /opt/soaps/store/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:716
> Event: 849026.684 Thread 0x00000008d7562800 Threw 0x00000008c2f32dd0 at /opt/soaps/store/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:716
> Event: 849026.877 Thread 0x00000008d7562800 Threw 0x00000008c4c4d830 at /opt/soaps/store/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:716
> Event: 849027.071 Thread 0x00000008d7562800 Threw 0x00000008c4c56310 at /opt/soaps/store/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:716
> Event: 849092.712 Thread 0x00000008e3049000 Threw 0x00000008a1fb2c70 at /opt/soaps/store/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jvm.cpp:2888
> Event: 849093.802 Thread 0x00000008d757b800 Threw 0x00000008a4651698 at /opt/soaps/store/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:716
> Event: 849093.824 Thread 0x00000008d757e800 Threw 0x00000008a47486b8 at /opt/soaps/store/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:716
> Event: 849093.878 Thread 0x00000008d757c800 Threw 0x00000008a4b40470 at /opt/soaps/store/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:716
> 
> Events (10 events):
> Event: 849093.721 Executing VM operation: RevokeBias done
> Event: 849093.731 Thread 0x00000008da2da800 Thread added: 0x00000008da2da800
> Event: 849093.738 Thread 0x00000008e3048800 Thread added: 0x00000008e3048800
> Event: 849105.145 Thread 0x00000008d9ee3800 Thread added: 0x00000008d9ee3800
> Event: 849105.145 Executing VM operation: RevokeBias
> Event: 849105.147 Executing VM operation: RevokeBias done
> Event: 849111.610 Thread 0x00000008d9ee3800 Thread exited: 0x00000008d9ee3800
> Event: 849111.979 Executing VM operation: ParallelGCFailedAllocation
> Event: 849111.994 Executing VM operation: ParallelGCFailedAllocation done
> Event: 849120.499 Thread 0x0000000800d52800 Thread added: 0x0000000800d52800
> 
> 
> Dynamic libraries:
> 0x0000000000400000 	/usr/local/openjdk7/bin/java
> 0x0000000800651000 	/lib/libz.so.5
> 0x0000000800766000 	/lib/libthr.so.3
> 0x000000080087f000 	/lib/libc.so.7
> 0x0000000800e00000 	/usr/local/openjdk7/jre/lib/amd64/server/libjvm.so
> 0x0000000801a31000 	/usr/lib/libstdc++.so.6
> 0x0000000800ac1000 	/lib/libm.so.5
> 0x0000000801c41000 	/lib/libgcc_s.so.1
> 0x0000000802000000 	/usr/local/openjdk7/jre/lib/amd64/libverify.so
> 0x000000080210f000 	/usr/local/openjdk7/jre/lib/amd64/libjava.so
> 0x0000000802235000 	/usr/local/openjdk7/jre/lib/amd64/libzip.so
> 0x00000008d8600000 	/usr/local/openjdk7/jre/lib/amd64/libnio.so
> 0x00000008d870f000 	/usr/local/openjdk7/jre/lib/amd64/libnet.so
> 0x00000008d8823000 	/usr/local/openjdk7/jre/lib/amd64/libmanagement.so
> 0x00000008d8a3d000 	/usr/local/lib/libtcnative-1.so.1
> 0x00000008d9400000 	/usr/lib/libssl.so.6
> 0x00000008d9553000 	/lib/libcrypto.so.6
> 0x00000008d97f3000 	/usr/local/lib/libapr-1.so.4
> 0x00000008d9920000 	/lib/libcrypt.so.5
> 0x00000008d9a39000 	/usr/local/openjdk7/jre/lib/amd64/libsunec.so
> 0x00000008ddc00000 	/usr/local/openjdk7/jre/lib/amd64/librmi.so
> 0x00000008e1c00000 	/usr/local/openjdk7/jre/lib/amd64/libawt.so
> 0x00000008deae1000 	/usr/local/openjdk7/jre/lib/amd64/headless/libmawt.so
> 0x00000008e6e00000 	/usr/local/openjdk7/jre/lib/amd64/libjpeg.so
> 0x00000008e7c00000 	/usr/local/openjdk7/jre/lib/amd64/liblcms.so
> 0x000000080050b000 	/libexec/ld-elf.so.1
> 
> VM Arguments:
> jvm_args: -Dversion=enterpriseedition -Xmx3076m -Xms256m -XX:PermSize=128M -XX:MaxPermSize=128M -Dfile.encoding=UTF-8 -Dlogback.ContextSelector=com.stimulus.archiva.log.WebAppLoggingContextSelector -Dversion=enterpriseedition -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/usr/local/mailarchiva/server/endorsed -Dcatalina.home=/usr/local/mailarchiva/server -Dcatalina.base=/usr/local/mailarchiva/server -Djava.util.logging.config.file=/usr/local/mailarchiva/server/conf/logging.properties -Djava.io.tmpdir=/usr/local/mailarchiva/server/temp 
> java_command: org.apache.catalina.startup.Bootstrap start
> Launcher Type: SUN_STANDARD
> 
> Environment Variables:
> JAVA_HOME=/usr/local/openjdk7
> PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/home/spoil/bin:/usr/local/mailarchiva/jre/bin
> SHELL=/usr/local/bin/bash
> 
> Signal Handlers:
> SIGSEGV: [libjvm.so+0x8e11c0], sa_mask[0]=0xfffefeff, sa_flags=0x00000042
> SIGBUS: [libjvm.so+0x8e11c0], sa_mask[0]=0xfffefeff, sa_flags=0x00000042
> SIGFPE: [libjvm.so+0x787670], sa_mask[0]=0xfffefeff, sa_flags=0x00000042
> SIGPIPE: [libjvm.so+0x787670], sa_mask[0]=0xfffefeff, sa_flags=0x00000042
> SIGXFSZ: [libjvm.so+0x787670], sa_mask[0]=0xfffefeff, sa_flags=0x00000042
> SIGILL: [libjvm.so+0x787670], sa_mask[0]=0xfffefeff, sa_flags=0x00000042
> SIGUSR1: SIG_DFL, sa_mask[0]=0x63807efb, sa_flags=0x00000000
> SIGUSR2: [libjvm.so+0x78a010], sa_mask[0]=0x00000000, sa_flags=0x00000042
> SIGHUP: [libjvm.so+0x7889d0], sa_mask[0]=0xfffefeff, sa_flags=0x00000042
> SIGINT: [libjvm.so+0x7889d0], sa_mask[0]=0xfffefeff, sa_flags=0x00000042
> SIGTERM: [libjvm.so+0x7889d0], sa_mask[0]=0xfffefeff, sa_flags=0x00000042
> SIGQUIT: [libjvm.so+0x7889d0], sa_mask[0]=0xfffefeff, sa_flags=0x00000042
> 
> 
> ---------------  S Y S T E M  ---------------
> 
> OS:BSDuname:FreeBSD 8.2-RELEASE-p3 FreeBSD 8.2-RELEASE-p3 #0: Tue Sep 27 18:45:57 UTC 2011     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
> rlimit: STACK 524288k, CORE infinity, NPROC 5547, NOFILE 11095, AS infinity
> load average:0.12 0.10 0.05
> 
> CPU:total 4 (4 cores per cpu, 1 threads per core) family 16 model 2 stepping 10, cmov, cx8, fxsr, mmx, sse, sse2, sse3, popcnt, mmxext, 3dnowpref, lzcnt, sse4a, tsc, tscinvbit, tscinv
> 
> /proc/cpuinfo:
>   <Not Available>
> 
> Memory: 4k page, physical 3531680k(882920k free)
> 
> /proc/meminfo:
> 
> 
> vm_info: OpenJDK 64-Bit Server VM (23.2-b09) for bsd-amd64 JRE (1.7.0_06-b24), built on Oct 17 2012 14:23:23 by "spoil" with gcc 4.2.1 20070719  [FreeBSD]
> 
> time: Sun Oct 28 08:01:54 2012
> elapsed time: 849120 seconds
> 
> > G'day Valentin,
> > 
> > On Fri, Oct 26, 2012 at 08:50:26AM +0400, Valentin Popov wrote:
> >> Using tomcat 7.0.32 for my email archiving application, and very often get this message with different problematic frame.  (no such things was on openjdk6) 
> > 
> > I don't suppose you have a simple test case that can reproduce it?
> > 
> >> #
> >> # A fatal error has been detected by the Java Runtime Environment:
> >> #
> >> #  SIGSEGV (0xb) at pc=0x000000093b61820a, pid=3671, tid=39732215808
> >> #
> >> # JRE version: 7.0-b24
> >> # Java VM: OpenJDK 64-Bit Server VM (23.2-b09 mixed mode bsd-amd64 compressed oops)
> >> # Problematic frame:
> >> # C  [libnet.so+0x920a]  Java_java_net_SocketInputStream_socketRead0+0x9a
> >> #
> >> # Core dump written. Default location: /usr/local/mailarchiva/server/logs/core or core.3671
> >> #
> >> # If you would like to submit a bug report, please visit:
> >> #   http://bugreport.sun.com/bugreport/crash.jsp
> >> # The crash happened outside the Java Virtual Machine in native code.
> >> # See problematic frame for where to report the bug.
> >> #
> >> 
> >> ---------------  T H R E A D  ---------------
> >> 
> >> Current thread (0x000000093ad0a000):  JavaThread "org.subethamail.smtp.server.Session-/127.0.0.1:15027" [_thread_in_native, id=1077510144, stack(0x00007ffff2f33000,0x00007ffff3033000)]
> >> 
> >> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00000000bad0a1d8
> >> 
> >> Registers:
> >> RAX=0x0000000000000001, RBX=0x0000000000000001, RCX=0x0000000000000000, RDX=0x0000000801007400
> >> RSP=0x00007ffff3022570, RBP=0x00007ffff30325c0, RSI=0x0000000000000000, RDI=0x0000000935ff8880
> >> R8 =0x0000000000000001, R9 =0x00000000bb67b620, R10=0xfffffe01a6dab488, R11=0x0000000000000246
> >> R12=0x00000000000006df, R13=0x00000000bad0a1d8, R14=0x00007ffff3022590, R15=0x000000000000ea60
> >> RIP=0x000000093b61820a, EFLAGS=0x0000000000000001, ERR=0x0000000000000004
> >> TRAPNO=0x000000000000000c
> >> 
> >> Top of Stack: (sp=0x00007ffff3022570)
> >> 0x00007ffff3022570:   0000000009ae5920 00007ffff3022590
> >> 0x00007ffff3022580:   0000000009aa85b8 00007ffff30325e8
> >> 0x00007ffff3022590:   0000000000000045 0000000000000000
> >> 0x00007ffff30225a0:   0000000000000000 00000000099a4428
> >> 0x00007ffff30225b0:   0000000009ab02a8 0000000000000000
> >> 0x00007ffff30225c0:   0000000000000000 0000000009ab04a8
> >> 0x00007ffff30225d0:   0000000009ac1930 0000000000000000
> >> 0x00007ffff30225e0:   0000000009aa84e0 0000000000000000
> >> 0x00007ffff30225f0:   0000000000000000 0000000009a32338
> >> 0x00007ffff3022600:   00000000099702f0 0000000000000000
> >> 0x00007ffff3022610:   0000000000000000 0000000000000000
> >> 0x00007ffff3022620:   0000000000000000 0000000000000000
> >> 0x00007ffff3022630:   00000000000032c0 0000000009aa4638
> >> 0x00007ffff3022640:   0000000000000000 0000000009a32338
> >> 0x00007ffff3022650:   0000000000000000 0000000009a38360
> >> 0x00007ffff3022660:   0000000000000000 0000000000000000
> >> 0x00007ffff3022670:   0000000000000000 0000000000000000
> >> 0x00007ffff3022680:   0000000000000000 0000000000000000
> >> 0x00007ffff3022690:   0000000000000000 00000000099b8248
> >> 0x00007ffff30226a0:   0000000000000000 0000000009998e58
> >> 0x00007ffff30226b0:   0000000009a81b38 0000000009a80f20
> >> 0x00007ffff30226c0:   0000000000000000 0000000000000000
> >> 0x00007ffff30226d0:   0000000000000000 0000000000000000
> >> 0x00007ffff30226e0:   0000000000000000 0000000000000000
> >> 0x00007ffff30226f0:   0000000009a52a08 0000000000000000
> >> 0x00007ffff3022700:   0000000009a34138 0000000000000000
> >> 0x00007ffff3022710:   0000000009a3dab8 0000000000000000
> >> 0x00007ffff3022720:   0000000009986518 0000000000001aa8
> >> 0x00007ffff3022730:   0000000009998fe0 0000000000000000
> >> 0x00007ffff3022740:   0000000009a52a08 0000000000000000
> >> 0x00007ffff3022750:   0000000009a34138 0000000000000000
> >> 0x00007ffff3022760:   0000000000000000 0000000000000000 
> >> 
> >> Instructions: (pc=0x000000093b61820a)
> >> 0x000000093b6181ea:   89 c6 45 85 ff 75 61 48 63 d3 4c 89 f6 44 89 e7
> >> 0x000000093b6181fa:   e8 dd bc ff ff 83 f8 00 89 c3 0f 8e 8c 00 00 00
> >> 0x000000093b61820a:   49 8b 45 00 4d 89 f0 89 d9 8b 95 c4 ff fe ff 48
> >> 0x000000093b61821a:   8b b5 c8 ff fe ff 4c 89 ef ff 90 80 06 00 00 4c 
> >> 
> >> Register to memory mapping:
> >> 
> >> RAX=0x0000000000000001 is an unknown value
> >> RBX=0x0000000000000001 is an unknown value
> >> RCX=0x0000000000000000 is an unknown value
> >> RDX=0x0000000801007400 is an unknown value
> >> RSP=0x00007ffff3022570 is pointing into the stack for thread: 0x000000093ad0a000
> >> RBP=0x00007ffff30325c0 is pointing into the stack for thread: 0x000000093ad0a000
> >> RSI=0x0000000000000000 is an unknown value
> >> RDI=0x0000000935ff8880 is an unknown value
> >> R8 =0x0000000000000001 is an unknown value
> >> R9 =0x00000000bb67b620 is an unknown value
> >> R10=0xfffffe01a6dab488 is an unknown value
> >> R11=0x0000000000000246 is an unknown value
> >> R12=0x00000000000006df is an unknown value
> >> R13=0x00000000bad0a1d8 is an unknown value
> >> R14=0x00007ffff3022590 is pointing into the stack for thread: 0x000000093ad0a000
> >> R15=0x000000000000ea60 is an unknown value
> >> 
> >> 
> >> Stack: [0x00007ffff2f33000,0x00007ffff3033000],  sp=0x00007ffff3022570,  free space=957k
> >> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> >> C  [libnet.so+0x920a]  Java_java_net_SocketInputStream_socketRead0+0x9a
> >> J  java.net.SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I
> >> 
> >> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
> >> J  java.net.SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I
> >> J  java.net.SocketInputStream.read([BII)I
> >> J  org.subethamail.smtp.io.CRLFTerminatedReader.readLine()Ljava/lang/String;
> >> J  org.subethamail.smtp.server.Session.runCommandLoop()V
> >> j  org.subethamail.smtp.server.Session.run()V+139
> >> J  java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V
> >> j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
> >> j  java.lang.Thread.run()V+11
> >> v  ~StubRoutines::call_stub
> >> 
> >> ---------------  P R O C E S S  ---------------
> >> 
> >> Java Threads: ( => current thread )
> >> 0x00000009675a7000 JavaThread "Lucene Merge Thread #115" daemon [_thread_in_Java, id=1427340288, stack(0x00007ffffccd0000,0x00007ffffcdd0000)]
> >> 0x0000000955a0f000 JavaThread "ActiveMQ BrokerService[ROOT] Task-8" daemon [_thread_blocked, id=661979136, stack(0x00007ffffb9bd000,0x00007ffffbabd000)]
> >> 0x0000000940c9a800 JavaThread "Lucene Merge Thread #111" daemon [_thread_in_Java, id=1077494784, stack(0x00007ffffcfd3000,0x00007ffffd0d3000)]
> >> 0x00000009485c5800 JavaThread "ActiveMQ Session Task-42" [_thread_blocked, id=1215825920, stack(0x00007ffffd9dd000,0x00007ffffdadd000)]
> >> 0x00000009485c4800 JavaThread "ActiveMQ Session Task-41" [_thread_blocked, id=1070077952, stack(0x00007ffffdade000,0x00007ffffdbde000)]
> >> 0x00000009485c4000 JavaThread "ActiveMQ Session Task-40" [_thread_blocked, id=1070062592, stack(0x00007ffffdee2000,0x00007ffffdfe2000)]
> >> 0x00000009276f4000 JavaThread "net.java.truecommons.io.Streams$ReaderThread" daemon [_thread_blocked, id=1077483520, stack(0x00007ffffc7cb000,0x00007ffffc8cb000)]
> >> 0x00000009467af000 JavaThread "ActiveMQ VMTransport: vm://ROOT#1-6" daemon [_thread_blocked, id=1605727232, stack(0x00007ffffd5d9000,0x00007ffffd6d9000)]
> >> 0x0000000927701000 JavaThread "ActiveMQ Session Task-39" [_thread_blocked, id=1070063616, stack(0x00007ffffcdd1000,0x00007ffffced1000)]
> >> 0x0000000927700000 JavaThread "ActiveMQ Session Task-38" [_thread_blocked, id=661974016, stack(0x00007ffff3034000,0x00007ffff3134000)]
> >> 0x00000009276ff800 JavaThread "ActiveMQ Session Task-37" [_thread_blocked, id=1077477376, stack(0x00007ffffbcc0000,0x00007ffffbdc0000)]
> >> 0x00000009485c3000 JavaThread "ActiveMQ Session Task-36" [_thread_blocked, id=661977088, stack(0x00007ffffc5c9000,0x00007ffffc6c9000)]
> >> 0x00000009485c2800 JavaThread "ActiveMQ Session Task-35" [_thread_blocked, id=661978112, stack(0x00007ffffdde1000,0x00007ffffdee1000)]
> >> 0x00000009481d3800 JavaThread "net.java.truecommons.io.Streams$ReaderThread" daemon [_thread_blocked, id=1077508096, stack(0x00007ffff2c30000,0x00007ffff2d30000)]
> >> 0x00000009276fe800 JavaThread "ActiveMQ Session Task-34" [_thread_blocked, id=1077497856, stack(0x00007ffffd1d5000,0x00007ffffd2d5000)]
> >> 0x00000009485c1800 JavaThread "ActiveMQ Session Task-33" [_thread_blocked, id=1202367488, stack(0x00007ffff2a2e000,0x00007ffff2b2e000)]
> >> 0x00000009485c1000 JavaThread "ActiveMQ Session Task-32" [_thread_blocked, id=1077474304, stack(0x00007ffff262a000,0x00007ffff272a000)]
> >> 0x00000009485c0000 JavaThread "ActiveMQ Session Task-31" [_thread_blocked, id=1070038016, stack(0x00007ffffc6ca000,0x00007ffffc7ca000)]
> >> 0x00000009485bf800 JavaThread "ActiveMQ Session Task-30" [_thread_blocked, id=1077492736, stack(0x00007ffffc2c6000,0x00007ffffc3c6000)]
> >> 0x0000000947609800 JavaThread "net.java.truecommons.io.Streams$ReaderThread" daemon [_thread_blocked, id=1077502976, stack(0x00007ffffc9cd000,0x00007ffffcacd000)]
> >> 0x000000093c823000 JavaThread "net.java.truecommons.io.Streams$ReaderThread" daemon [_thread_blocked, id=1070074880, stack(0x00007ffff2b2f000,0x00007ffff2c2f000)]
> >> 0x000000093ad0c000 JavaThread "org.subethamail.smtp.server.Session-/127.0.0.1:17352" [_thread_blocked, id=1077498880, stack(0x00007ffffced2000,0x00007ffffcfd2000)]
> >> 0x000000093ad0b800 JavaThread "org.subethamail.smtp.server.Session-/127.0.0.1:38202" [_thread_blocked, id=1070060544, stack(0x00007ffffd7db000,0x00007ffffd8db000)]
> >> 0x000000093ad0a800 JavaThread "org.subethamail.smtp.server.Session-/127.0.0.1:17191" [_thread_blocked, id=1077500928, stack(0x00007ffffcace000,0x00007ffffcbce000)]
> >> =>0x000000093ad0a000 JavaThread "org.subethamail.smtp.server.Session-/127.0.0.1:15027" [_thread_in_native, id=1077510144, stack(0x00007ffff2f33000,0x00007ffff3033000)]
> >> 0x000000093ad09000 JavaThread "org.subethamail.smtp.server.Session-/127.0.0.1:32044" [_thread_blocked, id=1070068736, stack(0x00007ffff2d31000,0x00007ffff2e31000)]
> >> 0x000000093ad08800 JavaThread "org.subethamail.smtp.server.Session-/127.0.0.1:35597" [_thread_blocked, id=1077511168, stack(0x00007ffff2529000,0x00007ffff2629000)]
> >> 0x00000009485be800 JavaThread "ActiveMQ Session Task-29" [_thread_blocked, id=1077480448, stack(0x00007ffffd6da000,0x00007ffffd7da000)]
> >> 0x00000009485be000 JavaThread "ActiveMQ Session Task-28" [_thread_blocked, id=661980160, stack(0x00007ffffdbdf000,0x00007ffffdcdf000)]
> >> 0x00000009485bd000 JavaThread "ActiveMQ Session Task-27" [_thread_blocked, id=689345536, stack(0x00007ffffd3d7000,0x00007ffffd4d7000)]
> >> 0x00000009485bc800 JavaThread "ActiveMQ Session Task-26" [_thread_blocked, id=1070065664, stack(0x00007ffff363a000,0x00007ffff373a000)]
> >> 0x000000093f5f0000 JavaThread "ActiveMQ VMTransport: vm://ROOT#1-5" daemon [_thread_blocked, id=1070071808, stack(0x00007ffffc1c5000,0x00007ffffc2c5000)]
> >> 0x000000094709e800 JavaThread "net.java.truecommons.io.Streams$ReaderThread" daemon [_thread_blocked, id=1077499904, stack(0x00007ffffd0d4000,0x00007ffffd1d4000)]
> >> 0x00000009276fc800 JavaThread "ActiveMQ Session Task-23" [_thread_blocked, id=1070059520, stack(0x00007ffffd8dc000,0x00007ffffd9dc000)]
> >> 0x00000009276fa000 JavaThread "ActiveMQ Session Task-16" [_thread_blocked, id=661983232, stack(0x00007ffff272b000,0x00007ffff282b000)]
> >> 0x00000009276f9800 JavaThread "ActiveMQ Session Task-15" [_thread_blocked, id=661982208, stack(0x00007ffff282c000,0x00007ffff292c000)]
> >> 0x00000009276f8800 JavaThread "ActiveMQ Session Task-14" [_thread_blocked, id=661981184, stack(0x00007ffff292d000,0x00007ffff2a2d000)]
> >> 0x000000093f3bf800 JavaThread "net.java.truecommons.io.Streams$ReaderThread" daemon [_thread_in_Java, id=1059032064, stack(0x00007ffff2e32000,0x00007ffff2f32000)]
> >> 0x00000009276f3800 JavaThread "ActiveMQ Session Task-7" [_thread_blocked, id=661969920, stack(0x00007ffff3236000,0x00007ffff3336000)]
> >> 0x00000009276f2800 JavaThread "ActiveMQ Session Task-6" [_thread_blocked, id=661968896, stack(0x00007ffff3337000,0x00007ffff3437000)]
> >> 0x00000009276f2000 JavaThread "ActiveMQ Session Task-5" [_thread_blocked, id=661967872, stack(0x00007ffff3438000,0x00007ffff3538000)]
> >> 0x00000009479b8800 JavaThread "ActiveMQ BrokerService[ROOT] Task-6" daemon [_thread_blocked, id=1202165760, stack(0x00007ffff3539000,0x00007ffff3639000)]
> >> 0x00000009479e8800 JavaThread "net.java.truecommons.io.Streams$ReaderThread" daemon [_thread_blocked, id=987235328, stack(0x00007ffffe1e5000,0x00007ffffe2e5000)]
> >> 0x00000009475d9000 JavaThread "net.java.truecommons.io.Streams$ReaderThread" daemon [_thread_blocked, id=991763456, stack(0x00007ffffe2e6000,0x00007ffffe3e6000)]
> >> 0x00000009485b8800 JavaThread "ActiveMQ VMTransport: vm://ROOT#0-4" daemon [_thread_blocked, id=991766528, stack(0x00007ffffb3b7000,0x00007ffffb4b7000)]
> >> 0x000000094706e000 JavaThread "net.java.truecommons.io.Streams$ReaderThread" daemon [_thread_blocked, id=1208698880, stack(0x00007ffffacb0000,0x00007ffffadb0000)]
> >> 0x0000000948e1e000 JavaThread "net.java.truecommons.io.Streams$ReaderThread" daemon [_thread_blocked, id=1077519360, stack(0x00007ffffa9ad000,0x00007ffffaaad000)]
> >> 0x000000093ad07800 JavaThread "org.subethamail.smtp.server.Session-/127.0.0.1:62106" [_thread_blocked, id=987239424, stack(0x00007ffffb1b5000,0x00007ffffb2b5000)]
> >> 0x000000093ad07000 JavaThread "org.subethamail.smtp.server.Session-/127.0.0.1:19566" [_thread_blocked, id=987236352, stack(0x00007ffffb0b4000,0x00007ffffb1b4000)]
> >> 0x000000093ad06000 JavaThread "org.subethamail.smtp.server.Session-/127.0.0.1:38626" [_thread_blocked, id=1077517312, stack(0x00007ffffa8ac000,0x00007ffffa9ac000)]
> >> 0x000000093ad05800 JavaThread "org.subethamail.smtp.server.Session-/127.0.0.1:59501" [_thread_blocked, id=1077518336, stack(0x00007ffffb2b6000,0x00007ffffb3b6000)]
> >> 0x000000093ad04800 JavaThread "org.subethamail.smtp.server.Session-/127.0.0.1:18756" [_thread_blocked, id=1191906304, stack(0x00007ffffb5b9000,0x00007ffffb6b9000)]
> >> 0x00000009485b8000 JavaThread "ActiveMQ Session Task-4" [_thread_blocked, id=1077493760, stack(0x00007ffffb4b8000,0x00007ffffb5b8000)]
> >> 0x00000009479b8000 JavaThread "ActiveMQ BrokerService[ROOT] Task-5" daemon [_thread_blocked, id=1077514240, stack(0x00007ffffabaf000,0x00007ffffacaf000)]
> >> 0x000000093f5ee800 JavaThread "ActiveMQ VMTransport: vm://ROOT#0-3" daemon [_thread_blocked, id=1088694272, stack(0x00007ffffcbcf000,0x00007ffffcccf000)]
> >> 0x00000009481d3000 JavaThread "ActiveMQ VMTransport: vm://ROOT#1-3" daemon [_thread_blocked, id=1066861568, stack(0x00007ffffbfc3000,0x00007ffffc0c3000)]
> >> 0x000000093ad04000 JavaThread "org.subethamail.smtp.server.Session-/127.0.0.1:39268" [_thread_blocked, id=17937408, stack(0x00007ffff7a7e000,0x00007ffff7b7e000)]
> >> 0x000000093c200800 JavaThread "http-nio-8090-exec-10" daemon [_thread_blocked, id=1013998592, stack(0x00007ffff373b000,0x00007ffff383b000)]
> >> 0x00000009273d2000 JavaThread "http-nio-8090-exec-9" daemon [_thread_blocked, id=1292022784, stack(0x00007ffff383c000,0x00007ffff393c000)]
> >> 0x00000009273d1000 JavaThread "http-nio-8090-exec-8" daemon [_thread_blocked, id=1292019712, stack(0x00007ffff393d000,0x00007ffff3a3d000)]
> >> 0x00000009273d0800 JavaThread "http-nio-8090-exec-7" daemon [_thread_blocked, id=1292016640, stack(0x00007ffff3a3e000,0x00007ffff3b3e000)]
> >> 0x00000009273cf800 JavaThread "http-nio-8090-exec-6" daemon [_thread_blocked, id=1292015616, stack(0x00007ffff3b3f000,0x00007ffff3c3f000)]
> >> 0x00000009273cf000 JavaThread "http-nio-8090-exec-5" daemon [_thread_blocked, id=1292014592, stack(0x00007ffff3c40000,0x00007ffff3d40000)]
> >> 0x00000009273ce000 JavaThread "http-nio-8090-exec-4" daemon [_thread_blocked, id=1292013568, stack(0x00007ffff3d41000,0x00007ffff3e41000)]
> >> 0x00000009273cd800 JavaThread "http-nio-8090-exec-3" daemon [_thread_blocked, id=1292012544, stack(0x00007ffff3e42000,0x00007ffff3f42000)]
> >> 0x00000009273cc800 JavaThread "http-nio-8090-exec-2" daemon [_thread_blocked, id=1292011520, stack(0x00007ffff3f43000,0x00007ffff4043000)]
> >> 0x00000009273cc000 JavaThread "http-nio-8090-exec-1" daemon [_thread_blocked, id=1292010496, stack(0x00007ffff4044000,0x00007ffff4144000)]
> >> 0x0000000948e12000 JavaThread "net.java.truecommons.io.Streams$ReaderThread" daemon [_thread_blocked, id=1189183488, stack(0x00007ffff4145000,0x00007ffff4245000)]
> >> 0x00000009467a7800 JavaThread "_var_store_test_vol201210b committer" daemon [_thread_blocked, id=1218537472, stack(0x00007ffff4246000,0x00007ffff4346000)]
> >> 0x0000000801297800 JavaThread "ajp-apr-8009-AsyncTimeout" daemon [_thread_blocked, id=17958912, stack(0x00007ffff4347000,0x00007ffff4447000)]
> >> 0x0000000801297000 JavaThread "ajp-apr-8009-Acceptor-0" daemon [_thread_in_native, id=17955840, stack(0x00007ffff4448000,0x00007ffff4548000)]
> >> 0x0000000801296000 JavaThread "ajp-apr-8009-CometPoller-0" daemon [_thread_blocked, id=17952768, stack(0x00007ffff4549000,0x00007ffff4649000)]
> >> 0x0000000801295800 JavaThread "ajp-apr-8009-Poller-0" daemon [_thread_blocked, id=17949696, stack(0x00007ffff464a000,0x00007ffff474a000)]
> >> 0x0000000801294800 JavaThread "http-nio-8090-Acceptor-0" daemon [_thread_in_native, id=17946624, stack(0x00007ffff474b000,0x00007ffff484b000)]
> >> 0x0000000801294000 JavaThread "http-nio-8090-ClientPoller-1" daemon [_thread_in_native, id=17944576, stack(0x00007ffff484c000,0x00007ffff494c000)]
> >> 0x0000000801293000 JavaThread "http-nio-8090-ClientPoller-0" daemon [_thread_in_native, id=17941504, stack(0x00007ffff494d000,0x00007ffff4a4d000)]
> >> 0x0000000801292800 JavaThread "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon [_thread_blocked, id=17939456, stack(0x00007ffff4a4e000,0x00007ffff4b4e000)]
> >> 0x0000000949692800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1218518016, stack(0x00007ffff4b4f000,0x00007ffff4c4f000)]
> >> 0x0000000949691800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180417024, stack(0x00007ffff4c50000,0x00007ffff4d50000)]
> >> 0x0000000949691000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180413952, stack(0x00007ffff4d51000,0x00007ffff4e51000)]
> >> 0x0000000949690000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180410880, stack(0x00007ffff4e52000,0x00007ffff4f52000)]
> >> 0x000000094968f800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180407808, stack(0x00007ffff4f53000,0x00007ffff5053000)]
> >> 0x000000094968e800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180404736, stack(0x00007ffff5054000,0x00007ffff5154000)]
> >> 0x000000094968e000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180401664, stack(0x00007ffff5155000,0x00007ffff5255000)]
> >> 0x000000094968d000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180398592, stack(0x00007ffff5256000,0x00007ffff5356000)]
> >> 0x000000094968c800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180395520, stack(0x00007ffff5357000,0x00007ffff5457000)]
> >> 0x000000094968b800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180392448, stack(0x00007ffff5458000,0x00007ffff5558000)]
> >> 0x000000094968b000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_in_Java, id=1180389376, stack(0x00007ffff5559000,0x00007ffff5659000)]
> >> 0x000000094968a000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180386304, stack(0x00007ffff565a000,0x00007ffff575a000)]
> >> 0x0000000949689800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180383232, stack(0x00007ffff575b000,0x00007ffff585b000)]
> >> 0x0000000949688800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180380160, stack(0x00007ffff585c000,0x00007ffff595c000)]
> >> 0x0000000949688000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180377088, stack(0x00007ffff595d000,0x00007ffff5a5d000)]
> >> 0x0000000949687000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180374016, stack(0x00007ffff5a5e000,0x00007ffff5b5e000)]
> >> 0x0000000949686800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180370944, stack(0x00007ffff5b5f000,0x00007ffff5c5f000)]
> >> 0x0000000949685800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180367872, stack(0x00007ffff5c60000,0x00007ffff5d60000)]
> >> 0x0000000949685000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180364800, stack(0x00007ffff5d61000,0x00007ffff5e61000)]
> >> 0x0000000949684000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180361728, stack(0x00007ffff5e62000,0x00007ffff5f62000)]
> >> 0x0000000949683800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180358656, stack(0x00007ffff5f63000,0x00007ffff6063000)]
> >> 0x0000000949682800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1180355584, stack(0x00007ffff6064000,0x00007ffff6164000)]
> >> 0x0000000949682000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216576512, stack(0x00007ffff6165000,0x00007ffff6265000)]
> >> 0x0000000949681000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216573440, stack(0x00007ffff6266000,0x00007ffff6366000)]
> >> 0x0000000949680800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216570368, stack(0x00007ffff6367000,0x00007ffff6467000)]
> >> 0x000000094967f800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216567296, stack(0x00007ffff6468000,0x00007ffff6568000)]
> >> 0x000000094967f000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216564224, stack(0x00007ffff6569000,0x00007ffff6669000)]
> >> 0x000000094967e000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216561152, stack(0x00007ffff666a000,0x00007ffff676a000)]
> >> 0x000000094967d800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216558080, stack(0x00007ffff676b000,0x00007ffff686b000)]
> >> 0x000000094967c800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216555008, stack(0x00007ffff686c000,0x00007ffff696c000)]
> >> 0x000000094967c000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216551936, stack(0x00007ffff696d000,0x00007ffff6a6d000)]
> >> 0x000000094967b000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216548864, stack(0x00007ffff6a6e000,0x00007ffff6b6e000)]
> >> 0x000000094967a800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216545792, stack(0x00007ffff6b6f000,0x00007ffff6c6f000)]
> >> 0x0000000949679800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216542720, stack(0x00007ffff6c70000,0x00007ffff6d70000)]
> >> 0x0000000949679000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216539648, stack(0x00007ffff6d71000,0x00007ffff6e71000)]
> >> 0x0000000949678000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216536576, stack(0x00007ffff6e72000,0x00007ffff6f72000)]
> >> 0x0000000949677800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216533504, stack(0x00007ffff6f73000,0x00007ffff7073000)]
> >> 0x0000000949676800 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216530432, stack(0x00007ffff7074000,0x00007ffff7174000)]
> >> 0x0000000949676000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216527360, stack(0x00007ffff7175000,0x00007ffff7275000)]
> >> 0x0000000949675000 JavaThread "_var_store_test_vol201210b indexers" daemon [_thread_blocked, id=1216524288, stack(0x00007ffff7276000,0x00007ffff7376000)]
> >> 0x0000000940bac000 JavaThread "Comet[]" daemon [_thread_blocked, id=1189180416, stack(0x00007ffff7377000,0x00007ffff7477000)]
> >> 0x0000000949674800 JavaThread "net.java.truecommons.io.Streams$ReaderThread" daemon [_thread_blocked, id=1216519168, stack(0x00007ffff7478000,0x00007ffff7578000)]
> >> 0x0000000940bab000 JavaThread "org.subethamail.smtp.server.ServerThread *:8091" daemon [_thread_in_native, id=1088708608, stack(0x00007ffff7579000,0x00007ffff7679000)]
> >> 0x0000000940baa800 JavaThread "lcheck" daemon [_thread_blocked, id=1088706560, stack(0x00007ffff767a000,0x00007ffff777a000)]
> >> 0x0000000940ba9800 JavaThread "log" daemon [_thread_blocked, id=1088703488, stack(0x00007ffff777b000,0x00007ffff787b000)]
> >> 0x0000000940ba9000 JavaThread "pdfbox" daemon [_thread_blocked, id=1088700416, stack(0x00007ffff787c000,0x00007ffff797c000)]
> >> 0x0000000940ba8000 JavaThread "license" daemon [_thread_blocked, id=1088697344, stack(0x00007ffff797d000,0x00007ffff7a7d000)]
> >> 0x0000000940ba6800 JavaThread "auto update" daemon [_thread_blocked, id=1088691200, stack(0x00007ffff7b7f000,0x00007ffff7c7f000)]
> >> 0x0000000940ba6000 JavaThread "stubbing" daemon [_thread_blocked, id=1088688128, stack(0x00007ffff7c80000,0x00007ffff7d80000)]
> >> 0x0000000940ba5000 JavaThread "purge" daemon [_thread_blocked, id=1088685056, stack(0x00007ffff7d81000,0x00007ffff7e81000)]
> >> 0x0000000940ba4800 JavaThread "pool-51-thread-1" [_thread_blocked, id=1088681984, stack(0x00007ffff7e82000,0x00007ffff7f82000)]
> >> 0x0000000940ba3800 JavaThread "diskspace" daemon [_thread_blocked, id=1088678912, stack(0x00007ffff7f83000,0x00007ffff8083000)]
> >> 0x0000000940ba3000 JavaThread "config auto load" daemon [_thread_blocked, id=1088675840, stack(0x00007ffff8084000,0x00007ffff8184000)]
> >> 0x0000000940ba2000 JavaThread "volume status detection" daemon [_thread_blocked, id=1088672768, stack(0x00007ffff8185000,0x00007ffff8285000)]
> >> 0x0000000940ba1800 JavaThread "receive" daemon [_thread_blocked, id=1088670720, stack(0x00007ffff8286000,0x00007ffff8386000)]
> >> 0x0000000940ba0800 JavaThread "receive" daemon [_thread_blocked, id=1088667648, stack(0x00007ffff8387000,0x00007ffff8487000)]
> >> 0x0000000940ba0000 JavaThread "receive" daemon [_thread_blocked, id=1088666624, stack(0x00007ffff8488000,0x00007ffff8588000)]
> >> 0x00000009409db800 JavaThread "receive" daemon [_thread_blocked, id=1088665600, stack(0x00007ffff8589000,0x00007ffff8689000)]
> >> 0x00000009409da800 JavaThread "receive" daemon [_thread_blocked, id=1088663552, stack(0x00007ffff868a000,0x00007ffff878a000)]
> >> 0x00000009409da000 JavaThread "receive" daemon [_thread_blocked, id=1088660480, stack(0x00007ffff878b000,0x00007ffff888b000)]
> >> 0x00000009409d9000 JavaThread "receive" daemon [_thread_blocked, id=1088657408, stack(0x00007ffff888c000,0x00007ffff898c000)]
> >> 0x00000009409d8800 JavaThread "receive" daemon [_thread_blocked, id=1088656384, stack(0x00007ffff898d000,0x00007ffff8a8d000)]
> >> 0x00000009409d7800 JavaThread "receive" daemon [_thread_blocked, id=1088653312, stack(0x00007ffff8a8e000,0x00007ffff8b8e000)]
> >> 0x00000009409d7000 JavaThread "receive" daemon [_thread_blocked, id=1085926400, stack(0x00007ffff8b8f000,0x00007ffff8c8f000)]
> >> 0x00000009409d6000 JavaThread "receive" daemon [_thread_blocked, id=1085924352, stack(0x00007ffff8c90000,0x00007ffff8d90000)]
> >> 0x00000009409d5800 JavaThread "receive" daemon [_thread_blocked, id=1085921280, stack(0x00007ffff8d91000,0x00007ffff8e91000)]
> >> 0x00000009409d4800 JavaThread "receive" daemon [_thread_blocked, id=1085918208, stack(0x00007ffff8e92000,0x00007ffff8f92000)]
> >> 0x00000009409d4000 JavaThread "receive" daemon [_thread_blocked, id=1085915136, stack(0x00007ffff8f93000,0x00007ffff9093000)]
> >> 0x00000009409d3000 JavaThread "receive" daemon [_thread_blocked, id=1085912064, stack(0x00007ffff9094000,0x00007ffff9194000)]
> >> 0x00000009409d2800 JavaThread "receive" daemon [_thread_blocked, id=1085908992, stack(0x00007ffff9195000,0x00007ffff9295000)]
> >> 0x00000009409d1800 JavaThread "receive" daemon [_thread_blocked, id=1085907968, stack(0x00007ffff9296000,0x00007ffff9396000)]
> >> 0x00000009409d1000 JavaThread "receive" daemon [_thread_blocked, id=1085906944, stack(0x00007ffff9397000,0x00007ffff9497000)]
> >> 0x00000009409d0000 JavaThread "receive" daemon [_thread_in_vm, id=1085904896, stack(0x00007ffff9498000,0x00007ffff9598000)]
> >> 0x00000009409cf800 JavaThread "receive" daemon [_thread_blocked, id=1085901824, stack(0x00007ffff9599000,0x00007ffff9699000)]
> >> 0x00000009409ce800 JavaThread "receive" daemon [_thread_blocked, id=1085899776, stack(0x00007ffff969a000,0x00007ffff979a000)]
> >> 0x00000009409ce000 JavaThread "receive" daemon [_thread_blocked, id=1085898752, stack(0x00007ffff979b000,0x00007ffff989b000)]
> >> 0x00000009409cd000 JavaThread "receive" daemon [_thread_blocked, id=1085897728, stack(0x00007ffff989c000,0x00007ffff999c000)]
> >> 0x00000009409cc800 JavaThread "receive" daemon [_thread_blocked, id=1085894656, stack(0x00007ffff999d000,0x00007ffff9a9d000)]
> >> 0x00000009409cb800 JavaThread "receive" daemon [_thread_blocked, id=1085891584, stack(0x00007ffff9a9e000,0x00007ffff9b9e000)]
> >> 0x00000009409cb000 JavaThread "receive" daemon [_thread_blocked, id=1085888512, stack(0x00007ffff9b9f000,0x00007ffff9c9f000)]
> >> 0x00000009409ca000 JavaThread "receive" daemon [_thread_blocked, id=1085885440, stack(0x00007ffff9ca0000,0x00007ffff9da0000)]
> >> 0x00000009409c9800 JavaThread "receive" daemon [_thread_blocked, id=1085883392, stack(0x00007ffff9da1000,0x00007ffff9ea1000)]
> >> 0x00000009409c8800 JavaThread "receive" daemon [_thread_blocked, id=1085882368, stack(0x00007ffff9ea2000,0x00007ffff9fa2000)]
> >> 0x00000009409c8000 JavaThread "receive" daemon [_thread_blocked, id=1085881344, stack(0x00007ffff9fa3000,0x00007ffffa0a3000)]
> >> 0x00000009409c7000 JavaThread "receive" daemon [_thread_blocked, id=1085878272, stack(0x00007ffffa0a4000,0x00007ffffa1a4000)]
> >> 0x00000009409c6800 JavaThread "receive" daemon [_thread_blocked, id=1085877248, stack(0x00007ffffa1a5000,0x00007ffffa2a5000)]
> >> 0x00000009409c5800 JavaThread "receive" daemon [_thread_blocked, id=1085874176, stack(0x00007ffffa2a6000,0x00007ffffa3a6000)]
> >> 0x00000009409c5000 JavaThread "receive" daemon [_thread_blocked, id=1085871104, stack(0x00007ffffa3a7000,0x00007ffffa4a7000)]
> >> 0x00000009409c4000 JavaThread "receive" daemon [_thread_blocked, id=1085868032, stack(0x00007ffffa4a8000,0x00007ffffa5a8000)]
> >> 0x00000009409c3800 JavaThread "receive" daemon [_thread_blocked, id=1085864960, stack(0x00007ffffa5a9000,0x00007ffffa6a9000)]
> >> 0x00000009409c2800 JavaThread "receive" daemon [_thread_blocked, id=1077533696, stack(0x00007ffffa6aa000,0x00007ffffa7aa000)]
> >> 0x00000009409bf800 JavaThread "receive" daemon [_thread_blocked, id=1077530624, stack(0x00007ffffa7ab000,0x00007ffffa8ab000)]
> >> 0x000000093f5ee000 JavaThread "ActiveMQ BrokerService[ROOT] Task-4" daemon [_thread_blocked, id=1062214656, stack(0x00007ffffaaae000,0x00007ffffabae000)]
> >> 0x00000009409be000 JavaThread "receive" daemon [_thread_blocked, id=1077527552, stack(0x00007ffffadb1000,0x00007ffffaeb1000)]
> >> 0x00000009409bc000 JavaThread "receive" daemon [_thread_blocked, id=1077524480, stack(0x00007ffffaeb2000,0x00007ffffafb2000)]
> >> 0x000000093ffe6800 JavaThread "receive" daemon [_thread_blocked, id=1077521408, stack(0x00007ffffafb3000,0x00007ffffb0b3000)]
> >> 0x000000093ffdc800 JavaThread "diskspacechecker" daemon [_thread_blocked, id=1077515264, stack(0x00007ffffb6ba000,0x00007ffffb7ba000)]
> >> 0x000000093ffdf800 JavaThread "statusreport" daemon [_thread_blocked, id=1077506048, stack(0x00007ffffb7bb000,0x00007ffffb8bb000)]
> >> 0x000000093ffdd800 JavaThread "statusreport" daemon [_thread_blocked, id=1070058496, stack(0x00007ffffb8bc000,0x00007ffffb9bc000)]
> >> 0x000000092a14f000 JavaThread "RMI RenewClean-[127.0.0.1:46245]" daemon [_thread_blocked, id=705686528, stack(0x00007ffffbabe000,0x00007ffffbbbe000)]
> >> 0x0000000929517800 JavaThread "RMI Scheduler(0)" daemon [_thread_blocked, id=693526528, stack(0x00007ffffbbbf000,0x00007ffffbcbf000)]
> >> 0x0000000929516800 JavaThread "RMI Reaper" [_thread_blocked, id=693524480, stack(0x00007ffffbdc1000,0x00007ffffbec1000)]
> >> 0x0000000929516000 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=693523456, stack(0x00007ffffbec2000,0x00007ffffbfc2000)]
> >> 0x00000009409c2000 JavaThread "RMI TCP Accept-1099" daemon [_thread_in_native, id=1077512192, stack(0x00007ffffc0c4000,0x00007ffffc1c4000)]
> >> 0x00000009409c1000 JavaThread "ActiveMQ Broker[ROOT] Scheduler" daemon [_thread_blocked, id=1077509120, stack(0x00007ffffc3c7000,0x00007ffffc4c7000)]
> >> 0x000000093ffd0000 JavaThread "deleteHelper" daemon [_thread_blocked, id=1070061568, stack(0x00007ffffdce0000,0x00007ffffdde0000)]
> >> 0x000000093ffcc000 JavaThread "AsyncAppender-Worker-Thread-4" daemon [_thread_blocked, id=1070034944, stack(0x00007ffffdfe3000,0x00007ffffe0e3000)]
> >> 0x000000093ffcb800 JavaThread "AsyncAppender-Worker-Thread-3" daemon [_thread_blocked, id=1070035968, stack(0x00007ffffe0e4000,0x00007ffffe1e4000)]
> >> 0x0000000801291000 JavaThread "NioBlockingSelector.BlockPoller-1" daemon [_thread_in_native, id=17935360, stack(0x00007ffffe3e7000,0x00007ffffe4e7000)]
> >> 0x0000000801290000 JavaThread "GC Daemon" daemon [_thread_blocked, id=17933312, stack(0x00007ffffe4e8000,0x00007ffffe5e8000)]
> >> 0x000000080128f800 JavaThread "Service Thread" daemon [_thread_blocked, id=17921024, stack(0x00007ffffe6ea000,0x00007ffffe7ea000)]
> >> 0x000000080128e800 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=17917952, stack(0x00007ffffe7eb000,0x00007ffffe8eb000)]
> >> 0x000000080128e000 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=17111040, stack(0x00007ffffe8ec000,0x00007ffffe9ec000)]
> >> 0x000000080128d000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=17107968, stack(0x00007ffffe9ed000,0x00007ffffeaed000)]
> >> 0x000000080128c800 JavaThread "Finalizer" daemon [_thread_blocked, id=17102848, stack(0x00007ffffeaee000,0x00007ffffebee000)]
> >> 0x000000080128b800 JavaThread "Reference Handler" daemon [_thread_blocked, id=17099776, stack(0x00007ffffebef000,0x00007ffffecef000)]
> >> 0x000000080128b000 JavaThread "main" [_thread_in_native, id=16807936, stack(0x00007fffffafe000,0x00007fffffbfe000)]
> >> 
> >> Other Threads:
> >> 0x00000008013bf800 VMThread [stack: 0x00007ffffecf0000,0x00007ffffedf0000] [id=17096704]
> >> 0x00000008013c0000 WatcherThread [stack: 0x00007ffffe5e9000,0x00007ffffe6e9000] [id=17924096]
> >> 
> >> VM state:synchronizing (normal execution)
> >> 
> >> VM Mutex/Monitor currently owned by a thread:  ([mutex/lock_event])
> >> [0x0000000801033100] Safepoint_lock - owner thread: 0x00000008013bf800
> >> [0x0000000801033180] Threads_lock - owner thread: 0x00000008013bf800
> >> [0x0000000801033680] Heap_lock - owner thread: 0x0000000949685800
> >> 
> >> Heap
> >> PSYoungGen      total 1344640K, used 1314115K [0x00000008d0970000, 0x0000000925de0000, 0x0000000925ec0000)
> >> eden space 1291968K, 100% used [0x00000008d0970000,0x000000091f720000,0x000000091f720000)
> >> from space 52672K, 42% used [0x000000091f720000,0x0000000920cc0d40,0x0000000922a90000)
> >> to   space 52416K, 0% used [0x0000000922ab0000,0x0000000922ab0000,0x0000000925de0000)
> >> ParOldGen       total 546240K, used 288771K [0x0000000825ec0000, 0x0000000847430000, 0x00000008d0970000)
> >> object space 546240K, 52% used [0x0000000825ec0000,0x00000008378c0fa8,0x0000000847430000)
> >> PSPermGen       total 524288K, used 67609K [0x0000000805ec0000, 0x0000000825ec0000, 0x0000000825ec0000)
> >> object space 524288K, 12% used [0x0000000805ec0000,0x000000080a0c6500,0x0000000825ec0000)
> >> 
> >> Card table byte_map: [0x0000000925ec0000,0x00000009267c1000] byte_map_base: 0x0000000921e90a00
> >> 
> >> Polling page: 0x0000000800668000
> >> 
> >> Code Cache  [0x0000000802ea9000, 0x0000000803b79000, 0x0000000805ea9000)
> >> total_blobs=4789 nmethods=4304 adapters=436 free_code_cache=36277Kb largest_free_block=36912576
> >> 
> >> Compilation events (10 events):
> >> Event: 289.239 Thread 0x000000080128e000 4560             jespa.security.Properties::getProperty (46 bytes)
> >> Event: 289.256 Thread 0x000000080128e800 4561             jespa.security.Properties::getProperty (16 bytes)
> >> Event: 289.281 Thread 0x000000080128e000 nmethod 4560 0x0000000803afbb90 code [0x0000000803afbdc0, 0x0000000803afcf80]
> >> Event: 289.294 Thread 0x000000080128e800 nmethod 4561 0x0000000803aa4a90 code [0x0000000803aa4ca0, 0x0000000803aa5c10]
> >> Event: 289.794 Thread 0x000000080128e000 4562             org.apache.lucene.util.Counter$SerialCounter::<init> (5 bytes)
> >> Event: 289.795 Thread 0x000000080128e000 nmethod 4562 0x0000000803931c90 code [0x0000000803931dc0, 0x0000000803931e18]
> >> Event: 289.797 Thread 0x000000080128e000 4563             org.apache.lucene.util.Counter$SerialCounter::<init> (10 bytes)
> >> Event: 289.797 Thread 0x000000080128e000 nmethod 4563 0x0000000803932a90 code [0x0000000803932bc0, 0x0000000803932c18]
> >> Event: 292.980 Thread 0x000000080128e000 4564             java.util.regex.Pattern$BranchConn::match (11 bytes)
> >> Event: 292.981 Thread 0x000000080128e000 nmethod 4564 0x0000000803aea650 code [0x0000000803aea7a0, 0x0000000803aea838]
> >> 
> >> GC Heap History (10 events):
> >> Event: 287.757 GC heap before
> >> {Heap before GC invocations=199 (full 12):
> >> PSYoungGen      total 1344448K, used 1291712K [0x00000008d0970000, 0x0000000925ec0000, 0x0000000925ec0000)
> >> eden space 1291712K, 100% used [0x00000008d0970000,0x000000091f6e0000,0x000000091f6e0000)
> >> from space 52736K, 0% used [0x0000000922b40000,0x0000000922b40000,0x0000000925ec0000)
> >> to   space 53184K, 0% used [0x000000091f6e0000,0x000000091f6e0000,0x0000000922ad0000)
> >> ParOldGen       total 546240K, used 241384K [0x0000000825ec0000, 0x0000000847430000, 0x00000008d0970000)
> >> object space 546240K, 44% used [0x0000000825ec0000,0x0000000834a7a1f8,0x0000000847430000)
> >> PSPermGen       total 524288K, used 67601K [0x0000000805ec0000, 0x0000000825ec0000, 0x0000000825ec0000)
> >> object space 524288K, 12% used [0x0000000805ec0000,0x000000080a0c44e8,0x0000000825ec0000)
> >> Event: 287.767 GC heap after
> >> Heap after GC invocations=199 (full 12):
> >> PSYoungGen      total 1332864K, used 41103K [0x00000008d0970000, 0x0000000925ec0000, 0x0000000925ec0000)
> >> eden space 1291712K, 0% used [0x00000008d0970000,0x00000008d0970000,0x000000091f6e0000)
> >> from space 41152K, 99% used [0x000000091f6e0000,0x0000000921f03f20,0x0000000921f10000)
> >> to   space 53888K, 0% used [0x0000000922a20000,0x0000000922a20000,0x0000000925ec0000)
> >> ParOldGen       total 546240K, used 241384K [0x0000000825ec0000, 0x0000000847430000, 0x00000008d0970000)
> >> object space 546240K, 44% used [0x0000000825ec0000,0x0000000834a7a1f8,0x0000000847430000)
> >> PSPermGen       total 524288K, used 67601K [0x0000000805ec0000, 0x0000000825ec0000, 0x0000000825ec0000)
> >> object space 524288K, 12% used [0x0000000805ec0000,0x000000080a0c44e8,0x0000000825ec0000)
> >> }
> >> Event: 288.998 GC heap before
> >> {Heap before GC invocations=200 (full 12):
> >> PSYoungGen      total 1332864K, used 1332815K [0x00000008d0970000, 0x0000000925ec0000, 0x0000000925ec0000)
> >> eden space 1291712K, 100% used [0x00000008d0970000,0x000000091f6e0000,0x000000091f6e0000)
> >> from space 41152K, 99% used [0x000000091f6e0000,0x0000000921f03f20,0x0000000921f10000)
> >> to   space 53888K, 0% used [0x0000000922a20000,0x0000000922a20000,0x0000000925ec0000)
> >> ParOldGen       total 546240K, used 242395K [0x0000000825ec0000, 0x0000000847430000, 0x00000008d0970000)
> >> object space 546240K, 44% used [0x0000000825ec0000,0x0000000834b76fa8,0x0000000847430000)
> >> PSPermGen       total 524288K, used 67601K [0x0000000805ec0000, 0x0000000825ec0000, 0x0000000825ec0000)
> >> object space 524288K, 12% used [0x0000000805ec0000,0x000000080a0c4670,0x0000000825ec0000)
> >> Event: 289.011 GC heap after
> >> Heap after GC invocations=200 (full 12):
> >> PSYoungGen      total 1345472K, used 36483K [0x00000008d0970000, 0x0000000925ec0000, 0x0000000925ec0000)
> >> eden space 1291584K, 0% used [0x00000008d0970000,0x00000008d0970000,0x000000091f6c0000)
> >> from space 53888K, 67% used [0x0000000922a20000,0x0000000924dc0dd0,0x0000000925ec0000)
> >> to   space 52608K, 0% used [0x000000091f6c0000,0x000000091f6c0000,0x0000000922a20000)
> >> ParOldGen       total 546240K, used 258020K [0x0000000825ec0000, 0x0000000847430000, 0x00000008d0970000)
> >> object space 546240K, 47% used [0x0000000825ec0000,0x0000000835ab9058,0x0000000847430000)
> >> PSPermGen       total 524288K, used 67601K [0x0000000805ec0000, 0x0000000825ec0000, 0x0000000825ec0000)
> >> object space 524288K, 12% used [0x0000000805ec0000,0x000000080a0c4670,0x0000000825ec0000)
> >> }
> >> Event: 290.132 GC heap before
> >> {Heap before GC invocations=201 (full 12):
> >> PSYoungGen      total 1345472K, used 1328067K [0x00000008d0970000, 0x0000000925ec0000, 0x0000000925ec0000)
> >> eden space 1291584K, 100% used [0x00000008d0970000,0x000000091f6c0000,0x000000091f6c0000)
> >> from space 53888K, 67% used [0x0000000922a20000,0x0000000924dc0dd0,0x0000000925ec0000)
> >> to   space 52608K, 0% used [0x000000091f6c0000,0x000000091f6c0000,0x0000000922a20000)
> >> ParOldGen       total 546240K, used 259025K [0x0000000825ec0000, 0x0000000847430000, 0x00000008d0970000)
> >> object space 546240K, 47% used [0x0000000825ec0000,0x0000000835bb4498,0x0000000847430000)
> >> PSPermGen       total 524288K, used 67602K [0x0000000805ec0000, 0x0000000825ec0000, 0x0000000825ec0000)
> >> object space 524288K, 12% used [0x0000000805ec0000,0x000000080a0c49a0,0x0000000825ec0000)
> >> Event: 290.143 GC heap after
> >> Heap after GC invocations=201 (full 12):
> >> PSYoungGen      total 1344192K, used 21769K [0x00000008d0970000, 0x0000000925e60000, 0x0000000925ec0000)
> >> eden space 1291584K, 0% used [0x00000008d0970000,0x00000008d0970000,0x000000091f6c0000)
> >> from space 52608K, 41% used [0x000000091f6c0000,0x0000000920c02400,0x0000000922a20000)
> >> to   space 53056K, 0% used [0x0000000922a90000,0x0000000922a90000,0x0000000925e60000)
> >> ParOldGen       total 546240K, used 272604K [0x0000000825ec0000, 0x0000000847430000, 0x00000008d0970000)
> >> object space 546240K, 49% used [0x0000000825ec0000,0x00000008368f7338,0x0000000847430000)
> >> PSPermGen       total 524288K, used 67602K [0x0000000805ec0000, 0x0000000825ec0000, 0x0000000825ec0000)
> >> object space 524288K, 12% used [0x0000000805ec0000,0x000000080a0c49a0,0x0000000825ec0000)
> >> }
> >> Event: 291.377 GC heap before
> >> {Heap before GC invocations=202 (full 12):
> >> PSYoungGen      total 1344192K, used 1313353K [0x00000008d0970000, 0x0000000925e60000, 0x0000000925ec0000)
> >> eden space 1291584K, 100% used [0x00000008d0970000,0x000000091f6c0000,0x000000091f6c0000)
> >> from space 52608K, 41% used [0x000000091f6c0000,0x0000000920c02400,0x0000000922a20000)
> >> to   space 53056K, 0% used [0x0000000922a90000,0x0000000922a90000,0x0000000925e60000)
> >> ParOldGen       total 546240K, used 272678K [0x0000000825ec0000, 0x0000000847430000, 0x00000008d0970000)
> >> object space 546240K, 49% used [0x0000000825ec0000,0x0000000836909858,0x0000000847430000)
> >> PSPermGen       total 524288K, used 67602K [0x0000000805ec0000, 0x0000000825ec0000, 0x0000000825ec0000)
> >> object space 524288K, 12% used [0x0000000805ec0000,0x000000080a0c49a0,0x0000000825ec0000)
> >> Event: 291.386 GC heap after
> >> Heap after GC invocations=202 (full 12):
> >> PSYoungGen      total 1345024K, used 24684K [0x00000008d0970000, 0x0000000925e70000, 0x0000000925ec0000)
> >> eden space 1291968K, 0% used [0x00000008d0970000,0x00000008d0970000,0x000000091f720000)
> >> from space 53056K, 46% used [0x0000000922a90000,0x00000009242ab040,0x0000000925e60000)
> >> to   space 52672K, 0% used [0x000000091f720000,0x000000091f720000,0x0000000922a90000)
> >> ParOldGen       total 546240K, used 277347K [0x0000000825ec0000, 0x0000000847430000, 0x00000008d0970000)
> >> object space 546240K, 50% used [0x0000000825ec0000,0x0000000836d98c48,0x0000000847430000)
> >> PSPermGen       total 524288K, used 67602K [0x0000000805ec0000, 0x0000000825ec0000, 0x0000000825ec0000)
> >> object space 524288K, 12% used [0x0000000805ec0000,0x000000080a0c49a0,0x0000000825ec0000)
> >> }
> >> Event: 292.664 GC heap before
> >> {Heap before GC invocations=203 (full 12):
> >> PSYoungGen      total 1345024K, used 1316652K [0x00000008d0970000, 0x0000000925e70000, 0x0000000925ec0000)
> >> eden space 1291968K, 100% used [0x00000008d0970000,0x000000091f720000,0x000000091f720000)
> >> from space 53056K, 46% used [0x0000000922a90000,0x00000009242ab040,0x0000000925e60000)
> >> to   space 52672K, 0% used [0x000000091f720000,0x000000091f720000,0x0000000922a90000)
> >> ParOldGen       total 546240K, used 277430K [0x0000000825ec0000, 0x0000000847430000, 0x00000008d0970000)
> >> object space 546240K, 50% used [0x0000000825ec0000,0x0000000836dadbe8,0x0000000847430000)
> >> PSPermGen       total 524288K, used 67607K [0x0000000805ec0000, 0x0000000825ec0000, 0x0000000825ec0000)
> >> object space 524288K, 12% used [0x0000000805ec0000,0x000000080a0c5d68,0x0000000825ec0000)
> >> Event: 292.674 GC heap after
> >> Heap after GC invocations=203 (full 12):
> >> PSYoungGen      total 1344640K, used 22147K [0x00000008d0970000, 0x0000000925de0000, 0x0000000925ec0000)
> >> eden space 1291968K, 0% used [0x00000008d0970000,0x00000008d0970000,0x000000091f720000)
> >> from space 52672K, 42% used [0x000000091f720000,0x0000000920cc0d40,0x0000000922a90000)
> >> to   space 52416K, 0% used [0x0000000922ab0000,0x0000000922ab0000,0x0000000925de0000)
> >> ParOldGen       total 546240K, used 288770K [0x0000000825ec0000, 0x0000000847430000, 0x00000008d0970000)
> >> object space 546240K, 52% used [0x0000000825ec0000,0x00000008378c0840,0x0000000847430000)
> >> PSPermGen       total 524288K, used 67607K [0x0000000805ec0000, 0x0000000825ec0000, 0x0000000825ec0000)
> >> object space 524288K, 12% used [0x0000000805ec0000,0x000000080a0c5d68,0x0000000825ec0000)
> >> }
> >> 
> >> Deoptimization events (10 events):
> >> Event: 285.888 Thread 0x0000000940ba1800 Uncommon trap -34 fr.pc 0x0000000803b46c4c
> >> Event: 285.942 Thread 0x00000009409d2800 Uncommon trap -34 fr.pc 0x0000000803936adc
> >> Event: 285.942 Thread 0x00000009409d2800 Uncommon trap -34 fr.pc 0x0000000803936adc
> >> Event: 285.942 Thread 0x00000009409d2800 Uncommon trap -34 fr.pc 0x0000000803936adc
> >> Event: 285.942 Thread 0x00000009409d2800 Uncommon trap -34 fr.pc 0x0000000803936adc
> >> Event: 288.284 Thread 0x00000009409c6800 Uncommon trap -58 fr.pc 0x00000008037ec234
> >> Event: 288.485 Thread 0x00000009409da800 Uncommon trap -58 fr.pc 0x00000008037ec234
> >> Event: 289.104 Thread 0x00000009409d7800 Uncommon trap -58 fr.pc 0x00000008037ec234
> >> Event: 291.410 Thread 0x000000093ad0c000 Uncommon trap -83 fr.pc 0x00000008037a8a28
> >> Event: 291.412 Thread 0x000000093ffcc000 Uncommon trap -58 fr.pc 0x0000000803538fb0
> >> 
> >> Internal exceptions (10 events):
> >> Event: 293.065 Thread 0x000000093c823000 Threw 0x00000008d1d35318 at /tank/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:742
> >> Event: 293.070 Thread 0x000000093ad0b800 Threw 0x00000008ee462c18 at /tank/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jvm.cpp:1166
> >> Event: 293.070 Thread 0x000000093ad0b800 Threw 0x00000008ee4908f8 at /tank/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jvm.cpp:1166
> >> Event: 293.070 Thread 0x000000093ad0b800 Threw 0x00000008ee588838 at /tank/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jvm.cpp:1166
> >> Event: 293.070 Thread 0x000000093ad0b800 Threw 0x00000008ee5dce20 at /tank/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jvm.cpp:1166
> >> Event: 293.289 Thread 0x00000009409da000 Threw 0x00000008fc572e30 at /tank/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:716
> >> Event: 293.347 Thread 0x00000009409c8800 Threw 0x0000000900917c50 at /tank/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:716
> >> Event: 293.502 Thread 0x0000000940ba0000 Threw 0x000000090c65fd48 at /tank/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:716
> >> Event: 293.617 Thread 0x00000009409ce800 Threw 0x00000009165b3cd8 at /tank/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:716
> >> Event: 293.731 Thread 0x000000093ad0a000 Threw 0x000000091f3a2e20 at /tank/ports/java/openjdk7/work/openjdk/hotspot/src/share/vm/prims/jni.cpp:742
> >> 
> >> Events (10 events):
> >> Event: 293.070 loading class 0x0000000940fc9100 done
> >> Event: 293.070 loading class 0x00000009496d7040
> >> Event: 293.070 loading class 0x00000009496d7040 done
> >> Event: 293.071 Executing VM operation: RevokeBias
> >> Event: 293.071 Executing VM operation: RevokeBias done
> >> Event: 293.145 Thread 0x00000009675a7000 Thread added: 0x00000009675a7000
> >> Event: 293.145 Executing VM operation: RevokeBias
> >> Event: 293.145 Executing VM operation: RevokeBias done
> >> Event: 293.731 Executing VM operation: RevokeBias
> >> Event: 293.731 Executing VM operation: RevokeBias done
> >> 
> >> 
> >> Dynamic libraries:
> >> 0x0000000000400000 	/usr/local/openjdk7/bin/java
> >> 0x0000000800855000 	/lib/libz.so.6
> >> 0x0000000800a6b000 	/lib/libthr.so.3
> >> 0x0000000800c8e000 	/lib/libc.so.7
> >> 0x0000000801400000 	/usr/local/openjdk7/jre/lib/amd64/server/libjvm.so
> >> 0x000000080212f000 	/usr/lib/libstdc++.so.6
> >> 0x000000080243f000 	/lib/libm.so.5
> >> 0x0000000802660000 	/lib/libgcc_s.so.1
> >> 0x000000080286d000 	/usr/local/openjdk7/jre/lib/amd64/libverify.so
> >> 0x0000000802a7b000 	/usr/local/openjdk7/jre/lib/amd64/libjava.so
> >> 0x0000000802ca1000 	/usr/local/openjdk7/jre/lib/amd64/libzip.so
> >> 0x000000093b400000 	/usr/local/openjdk7/jre/lib/amd64/libnio.so
> >> 0x000000093b60f000 	/usr/local/openjdk7/jre/lib/amd64/libnet.so
> >> 0x000000093b823000 	/usr/local/openjdk7/jre/lib/amd64/libmanagement.so
> >> 0x000000093dc00000 	/usr/local/lib/libtcnative-1.so.1
> >> 0x000000093de23000 	/usr/lib/libssl.so.6
> >> 0x000000093e076000 	/lib/libcrypto.so.6
> >> 0x000000093e416000 	/usr/local/lib/libapr-1.so.4
> >> 0x000000093e642000 	/lib/libcrypt.so.5
> >> 0x000000093e862000 	/usr/local/openjdk7/jre/lib/amd64/libsunec.so
> >> 0x0000000947c00000 	/usr/local/openjdk7/jre/lib/amd64/librmi.so
> >> 0x000000080060c000 	/libexec/ld-elf.so.1
> >> 
> >> VM Arguments:
> >> jvm_args: -Xmx4096m -Xms256m -XX:PermSize=512M -XX:MaxPermSize=512M -Dfile.encoding=UTF-8 -Dlogback.ContextSelector=com.stimulus.archiva.log.WebAppLoggingContextSelector -Dversion=enterpriseedition -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/usr/local/mailarchiva/server/endorsed -Dcatalina.home=/usr/local/mailarchiva/server -Dcatalina.base=/usr/local/mailarchiva/server -Djava.util.logging.config.file=/usr/local/mailarchiva/server/conf/logging.properties -Djava.io.tmpdir=/usr/local/mailarchiva/server/temp 
> >> java_command: org.apache.catalina.startup.Bootstrap start
> >> Launcher Type: SUN_STANDARD
> >> 
> >> Environment Variables:
> >> JAVA_HOME=/usr/local/openjdk7
> >> PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/root/bin
> >> SHELL=/bin/csh
> >> HOSTTYPE=FreeBSD
> >> OSTYPE=FreeBSD
> >> MACHTYPE=unknown
> >> 
> >> Signal Handlers:
> >> SIGSEGV: [libjvm.so+0x8e0770], sa_mask[0]=0x7fffffff, sa_flags=0x00000042
> >> SIGBUS: [libjvm.so+0x8e0770], sa_mask[0]=0x7fffffff, sa_flags=0x00000042
> >> SIGFPE: [libjvm.so+0x786900], sa_mask[0]=0x7fffffff, sa_flags=0x00000042
> >> SIGPIPE: [libjvm.so+0x786900], sa_mask[0]=0x7fffffff, sa_flags=0x00000042
> >> SIGXFSZ: [libjvm.so+0x786900], sa_mask[0]=0x7fffffff, sa_flags=0x00000042
> >> SIGILL: [libjvm.so+0x786900], sa_mask[0]=0x7fffffff, sa_flags=0x00000042
> >> SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
> >> SIGUSR2: [libjvm.so+0x7892a0], sa_mask[0]=0x00000000, sa_flags=0x00000042
> >> SIGHUP: [libjvm.so+0x787c60], sa_mask[0]=0x7fffffff, sa_flags=0x00000042
> >> SIGINT: [libjvm.so+0x787c60], sa_mask[0]=0x7fffffff, sa_flags=0x00000042
> >> SIGTERM: [libjvm.so+0x787c60], sa_mask[0]=0x7fffffff, sa_flags=0x00000042
> >> SIGQUIT: [libjvm.so+0x787c60], sa_mask[0]=0x7fffffff, sa_flags=0x00000042
> >> 
> >> 
> >> ---------------  S Y S T E M  ---------------
> >> 
> >> OS:BSDuname:FreeBSD 9.0-RELEASE-p1 FreeBSD 9.0-RELEASE-p1 #0: Thu May 24 13:34:40 MSK 2012     root@boss:/usr/obj/usr/src/sys/GENERIC amd64
> >> rlimit: STACK 524288k, CORE infinity, NPROC 5547, NOFILE 184320, AS infinity
> >> load average:19.57 11.99 7.80
> >> 
> >> CPU:total 16 (4 cores per cpu, 2 threads per core) family 6 model 44 stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, ht, tsc, tscinvbit, tscinv
> >> 
> >> /proc/cpuinfo:
> >> <Not Available>
> >> 
> >> Memory: 4k page, physical 25963636k(6490909k free)
> >> 
> >> /proc/meminfo:
> >> 
> >> 
> >> vm_info: OpenJDK 64-Bit Server VM (23.2-b09) for bsd-amd64 JRE (1.7.0_06-b24), built on Oct 19 2012 13:15:45 by "ma" with gcc 4.2.1 20070831 patched [FreeBSD]
> >> 
> >> time: Wed Oct 24 11:10:14 2012
> >> elapsed time: 293 seconds
> >> 
> >> 
> >> thanks
> >> Regards
> >> Valentin
> >> 
> >> 
> >> 
> >> 
> >> 
> >> _______________________________________________
> >> freebsd-java@freebsd.org mailing list
> >> http://lists.freebsd.org/mailman/listinfo/freebsd-java
> >> To unsubscribe, send any mail to "freebsd-java-unsubscribe@freebsd.org"
> > 
> > -- 
> > Greg Lewis                          Email   : glewis@eyesbeyond.com
> > Eyes Beyond                         Web     : http://www.eyesbeyond.com
> > Information Technology              FreeBSD : glewis@FreeBSD.org
> 
> 
>  ?? ??????????????????,
> ???????????????? ??????????
> 
> 
> 
> 
> 

-- 
Greg Lewis                          Email   : glewis@eyesbeyond.com
Eyes Beyond                         Web     : http://www.eyesbeyond.com
Information Technology              FreeBSD : glewis@FreeBSD.org



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20121029201710.GA12518>