When Weblogic gives an out of memory error, check the following things:
Weblogic admin gui: mydomain2 > servers
Determine which server is lowest on memory. Is this the server that is running the Inetsoft scheduler?
grep for errors in reedi:/usr/local/bea81/user_projects/mydomain2/mydomain2.log
Look for out of memory errors or stuck thread errors.
How do I get a thread dump from a hung java process?
When we redeploy our web apps through a script, it occasionally hangs the admin server. This is a bug that BEA fixed in SP3 but we have not applied.
kill -3 <pid>
This will cause the process to write out the state of its threads to STDOUT, which we have directed to nohup.out.
The output from a hung admin server looks like this:
Full thread dump Java HotSpot(TM) Server VM (1.4.1_02-ea-b01 mixed mode):
"ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" daemon prio=5 tid=0xa0f490 nid=0x30 runnable [dc9ff000..dc9ffc24]
at weblogic.socket.PosixSocketMuxer.poll(Native Method)
at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:99)
- locked <eab9e280> (a java.lang.String)
at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:178)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:151)
"ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" daemon prio=5 tid=0x749490 nid=0x2f waiting for monitor entry [dcaff000..dcaffc24]
at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:90)
- waiting to lock <eab9e280> (a java.lang.String)
at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:178)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:151)
"ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" daemon prio=5 tid=0xa8e0b8 nid=0x2e waiting for monitor entry [dcbff000..dcbffc24]
at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:90)
- waiting to lock <eab9e280> (a java.lang.String)
at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:178)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:151)
"ListenThread.Default" prio=5 tid=0x2b7dd0 nid=0x2d runnable [dccff000..dccffc24]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
- locked <eaba3358> (a java.net.PlainSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:439)
at java.net.ServerSocket.accept(ServerSocket.java:410)
at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:24)
at weblogic.t3.srvr.ListenThread.accept(ListenThread.java:713)
at weblogic.t3.srvr.ListenThread.run(ListenThread.java:290)
"weblogic.health.CoreHealthMonitor" daemon prio=5 tid=0x1036580 nid=0x2b waiting on condition [de17f000..de17fc24]
at java.lang.Thread.sleep(Native Method)
at weblogic.t3.srvr.CoreHealthMonitorThread.run(CoreHealthMonitorThread.java:128)
"Thread-4" prio=5 tid=0xa0fd58 nid=0x2a in Object.wait() [de27f000..de27fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea8a9520> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:429)
- locked <ea8a9520> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:382)
"LDAPConnThread ldap.psmfc.org:389" daemon prio=5 tid=0x907ba0 nid=0x28 runnable [de47f000..de47fc24]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
- locked <ea7ab250> (a java.io.BufferedInputStream)
at netscape.ldap.ber.stream.BERElement.getElement(BERElement.java:101)
at netscape.ldap.LDAPConnThread.run(LDAPConnThread.java:420)
"VDE Replication Thread" prio=5 tid=0xdf26c0 nid=0x26 in Object.wait() [de57f000..de57fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea7ab278> (a com.octetstring.vde.replication.Replication)
at com.octetstring.vde.replication.Replication.wait30sec(Replication.java:395)
- locked <ea7ab278> (a com.octetstring.vde.replication.Replication)
at com.octetstring.vde.replication.Replication.run(Replication.java:365)
"VDE Transaction Processor Thread" prio=2 tid=0x5ddb28 nid=0x25 in Object.wait() [de67f000..de67fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea7ab2d8> (a com.octetstring.vde.backend.standard.TransactionProcessor)
at java.lang.Object.wait(Object.java:426)
at com.octetstring.vde.backend.standard.TransactionProcessor.waitChange(TransactionProcessor.java:355)
- locked <ea7ab2d8> (a com.octetstring.vde.backend.standard.TransactionProcessor)
at com.octetstring.vde.backend.standard.TransactionProcessor.run(TransactionProcessor.java:215)
"ExecuteThread: '2' for queue: 'weblogic.admin.RMI'" daemon prio=5 tid=0x7b5078 nid=0x24 in Object.wait() [de77f000..de77fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71e5a0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71e5a0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '1' for queue: 'weblogic.admin.RMI'" daemon prio=5 tid=0x624bb0 nid=0x23 in Object.wait() [de87f000..de87fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71e620> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71e620> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '0' for queue: 'weblogic.admin.RMI'" daemon prio=5 tid=0x9fb1b0 nid=0x22 in Object.wait() [de97f000..de97fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71e6a0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71e6a0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '1' for queue: 'weblogic.admin.HTTP'" daemon prio=5 tid=0x3f8458 nid=0x21 in Object.wait() [dea7f000..dea7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71e720> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71e720> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '0' for queue: 'weblogic.admin.HTTP'" daemon prio=5 tid=0x9fb680 nid=0x20 in Object.wait() [deb7f000..deb7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71e7a0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71e7a0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"weblogic.security.SpinnerRandomSource" daemon prio=5 tid=0x9fb4e8 nid=0x1f in Object.wait() [dec7f000..dec7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71e868> (a java.lang.Object)
at java.lang.Object.wait(Object.java:426)
at weblogic.security.SpinnerRandomBitsSource.run(SpinnerRandomBitsSource.java:60)
- locked <ea71e868> (a java.lang.Object)
at java.lang.Thread.run(Thread.java:536)
"weblogic.time.TimeEventGenerator" daemon prio=9 tid=0x3f73b0 nid=0x1e in Object.wait() [ded7f000..ded7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71e8d8> (a weblogic.time.common.internal.TimeTable)
at weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:271)
- locked <ea71e8d8> (a weblogic.time.common.internal.TimeTable)
at weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java:118)
at java.lang.Thread.run(Thread.java:536)
"ExecuteThread: '4' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x3f6760 nid=0x1d in Object.wait() [dee7f000..dee7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71e950> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71e950> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '3' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x3f5c50 nid=0x1c in Object.wait() [def7f000..def7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71e9d0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71e9d0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '2' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x77ad98 nid=0x1b in Object.wait() [df07f000..df07fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71ea50> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71ea50> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '1' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x5ab5e0 nid=0x1a in Object.wait() [df17f000..df17fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71ead0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71ead0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '0' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x5aab08 nid=0x19 in Object.wait() [df27f000..df27fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71eb50> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71eb50> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '14' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x82c868 nid=0x18 in Object.wait() [df37f000..df37fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71ebd0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71ebd0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '13' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x82bd20 nid=0x17 in Object.wait() [df47f000..df47fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71ec50> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71ec50> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '12' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x865a60 nid=0x16 in Object.wait() [df57f000..df57fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71ecd0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71ecd0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '11' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x864ee0 nid=0x15 in Object.wait() [df67f000..df67fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71ed50> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71ed50> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '10' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x3b0760 nid=0x14 in Object.wait() [df77f000..df77fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71edd0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71edd0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '9' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x50c598 nid=0x13 in Object.wait() [df87f000..df87fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71ee50> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71ee50> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '8' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x56aa80 nid=0x12 in Object.wait() [df97f000..df97fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71eed0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71eed0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '7' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x50c360 nid=0x11 in Object.wait() [dfa7f000..dfa7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71ef50> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71ef50> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '6' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x2cefd8 nid=0x10 in Object.wait() [dfb7f000..dfb7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71efd0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71efd0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '5' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x2cee40 nid=0xf in Object.wait() [e057f000..e057fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71f050> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71f050> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '4' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x2d0088 nid=0xe in Object.wait() [e067f000..e067fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71f0d0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71f0d0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '3' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x8692a8 nid=0xd in Object.wait() [e077f000..e077fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71f150> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71f150> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '2' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x7227a0 nid=0xc in Object.wait() [e247f000..e247fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71f1d0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71f1d0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '1' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x50cd98 nid=0xb in Object.wait() [e257f000..e257fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71f250> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71f250> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '0' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0xb68be8 nid=0xa in Object.wait() [e267f000..e267fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea71f2d0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea71f2d0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"Signal Dispatcher" daemon prio=10 tid=0xdfca0 nid=0x7 waiting on condition [0..0]
"Finalizer" daemon prio=8 tid=0xda070 nid=0x4 in Object.wait() [fa97f000..fa97fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea3e3398> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:111)
- locked <ea3e3398> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:127)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=10 tid=0xd8f18 nid=0x3 in Object.wait() [faa7f000..faa7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea3e3400> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:426)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:113)
- locked <ea3e3400> (a java.lang.ref.Reference$Lock)
"main" prio=5 tid=0x2d0c0 nid=0x1 in Object.wait() [ffbfe000..ffbfe5e4]
at java.lang.Object.wait(Native Method)
- waiting on <ea3e3430> (a weblogic.t3.srvr.T3Srvr)
at java.lang.Object.wait(Object.java:426)
at weblogic.t3.srvr.T3Srvr.waitForDeath(T3Srvr.java:1135)
- locked <ea3e3430> (a weblogic.t3.srvr.T3Srvr)
at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:379)
at weblogic.Server.main(Server.java:32)
"VM Thread" prio=5 tid=0xd8898 nid=0x2 runnable
"VM Periodic Task Thread" prio=10 tid=0xdea00 nid=0x5 waiting on condition
"Suspend Checker Thread" prio=10 tid=0xdf350 nid=0x6 runnable
A thread dump on a freshly restarted admin server looks like this:
Full thread dump Java HotSpot(TM) Server VM (1.4.1_02-ea-b01 mixed mode):
"Thread-6" daemon prio=5 tid=0x7fc0d8 nid=0x32 runnable [de37f000..de37fc24]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:222)
at java.io.BufferedInputStream.read(BufferedInputStream.java:277)
- locked <e6accf58> (a java.io.BufferedInputStream)
at com.sun.jndi.ldap.Connection.run(Connection.java:779)
at java.lang.Thread.run(Thread.java:536)
"ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'" daemon prio=5 tid=0x589040 nid=0x30 waiting for monitor entry [dc9ff000..dc9ffc24]
at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:92)
- waiting to lock <eb160c40> (a java.lang.String)
at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:178)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:151)
"ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" daemon prio=5 tid=0xfd1bb0 nid=0x2f runnable [dcaff000..dcaffc24]
at weblogic.socket.PosixSocketMuxer.poll(Native Method)
at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:99)
- locked <eb160c40> (a java.lang.String)
at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:178)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:151)
"ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" daemon prio=5 tid=0xcc46a8 nid=0x2e waiting for monitor entry [dcbff000..dcbffc24]
at weblogic.socket.PosixSocketMuxer.processSockets(PosixSocketMuxer.java:92)
- waiting to lock <eb160c40> (a java.lang.String)
at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:178)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:151)
"ListenThread.Default" prio=5 tid=0x9c5948 nid=0x2d runnable [dccff000..dccffc24]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
- locked <eb160e40> (a java.net.PlainSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:439)
at java.net.ServerSocket.accept(ServerSocket.java:410)
at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:24)
at weblogic.t3.srvr.ListenThread.accept(ListenThread.java:713)
at weblogic.t3.srvr.ListenThread.run(ListenThread.java:290)
"Thread-5" daemon prio=5 tid=0xb40118 nid=0x2c runnable [de07f000..de07fc24]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:222)
at java.io.BufferedInputStream.read(BufferedInputStream.java:277)
- locked <eb09b628> (a java.io.BufferedInputStream)
at com.sun.jndi.ldap.Connection.run(Connection.java:779)
at java.lang.Thread.run(Thread.java:536)
"weblogic.health.CoreHealthMonitor" daemon prio=5 tid=0xada4f0 nid=0x2b waiting on condition [de17f000..de17fc24]
at java.lang.Thread.sleep(Native Method)
at weblogic.t3.srvr.CoreHealthMonitorThread.run(CoreHealthMonitorThread.java:126)
"Thread-4" prio=5 tid=0x6c7be0 nid=0x2a in Object.wait() [de27f000..de27fc24]
at java.lang.Object.wait(Native Method)
- waiting on <eaac72f8> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:429)
- locked <eaac72f8> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:382)
"LDAPConnThread ldap.psmfc.org:389" daemon prio=5 tid=0x808160 nid=0x28 runnable [de47f000..de47fc24]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
- locked <ea9a9398> (a java.io.BufferedInputStream)
at netscape.ldap.ber.stream.BERElement.getElement(BERElement.java:101)
at netscape.ldap.LDAPConnThread.run(LDAPConnThread.java:420)
"VDE Replication Thread" prio=5 tid=0x6c91b0 nid=0x26 in Object.wait() [de57f000..de57fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea9a93e8> (a com.octetstring.vde.replication.Replication)
at com.octetstring.vde.replication.Replication.wait30sec(Replication.java:395)
- locked <ea9a93e8> (a com.octetstring.vde.replication.Replication)
at com.octetstring.vde.replication.Replication.run(Replication.java:365)
"VDE Transaction Processor Thread" prio=2 tid=0x4f73e8 nid=0x25 in Object.wait() [de67f000..de67fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea9a9448> (a com.octetstring.vde.backend.standard.TransactionProcessor)
at java.lang.Object.wait(Object.java:426)
at com.octetstring.vde.backend.standard.TransactionProcessor.waitChange(TransactionProcessor.java:355)
- locked <ea9a9448> (a com.octetstring.vde.backend.standard.TransactionProcessor)
at com.octetstring.vde.backend.standard.TransactionProcessor.run(TransactionProcessor.java:215)
"ExecuteThread: '2' for queue: 'weblogic.admin.RMI'" daemon prio=5 tid=0x282570 nid=0x24 in Object.wait() [de77f000..de77fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90f520> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90f520> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '1' for queue: 'weblogic.admin.RMI'" daemon prio=5 tid=0x281dc0 nid=0x23 in Object.wait() [de87f000..de87fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90f5a0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90f5a0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '0' for queue: 'weblogic.admin.RMI'" daemon prio=5 tid=0x281c28 nid=0x22 in Object.wait() [de97f000..de97fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90f620> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90f620> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '1' for queue: 'weblogic.admin.HTTP'" daemon prio=5 tid=0x281a90 nid=0x21 in Object.wait() [dea7f000..dea7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90f6a0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90f6a0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '0' for queue: 'weblogic.admin.HTTP'" daemon prio=5 tid=0x2810f8 nid=0x20 in Object.wait() [deb7f000..deb7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90f720> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90f720> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"weblogic.security.SpinnerRandomSource" daemon prio=5 tid=0x5afc68 nid=0x1f in Object.wait() [dec7f000..dec7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90f7e8> (a java.lang.Object)
at java.lang.Object.wait(Object.java:426)
at weblogic.security.SpinnerRandomBitsSource.run(SpinnerRandomBitsSource.java:60)
- locked <ea90f7e8> (a java.lang.Object)
at java.lang.Thread.run(Thread.java:536)
"weblogic.time.TimeEventGenerator" daemon prio=9 tid=0x5c9c58 nid=0x1e in Object.wait() [ded7f000..ded7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90f858> (a weblogic.time.common.internal.TimeTable)
at weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:271)
- locked <ea90f858> (a weblogic.time.common.internal.TimeTable)
at weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java:118)
at java.lang.Thread.run(Thread.java:536)
"ExecuteThread: '4' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x53c150 nid=0x1d in Object.wait() [dee7f000..dee7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90f8d0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90f8d0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '3' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x53b5a0 nid=0x1c in Object.wait() [def7f000..def7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90f950> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90f950> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '2' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x53ab68 nid=0x1b in Object.wait() [df07f000..df07fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90f9d0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90f9d0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '1' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x527130 nid=0x1a in Object.wait() [df17f000..df17fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90fa50> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90fa50> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '0' for queue: 'weblogic.kernel.System'" daemon prio=5 tid=0x526580 nid=0x19 in Object.wait() [df27f000..df27fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90fad0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90fad0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '14' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x5259d0 nid=0x18 in Object.wait() [df37f000..df37fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90fb50> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90fb50> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '13' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x524e88 nid=0x17 in Object.wait() [df47f000..df47fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90fbd0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90fbd0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '12' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x55c070 nid=0x16 in Object.wait() [df57f000..df57fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90fc50> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90fc50> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '11' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x55b4c0 nid=0x15 in Object.wait() [df67f000..df67fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90fcd0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90fcd0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '10' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x55a978 nid=0x14 in Object.wait() [df77f000..df77fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90fd50> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90fd50> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '9' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x559ea0 nid=0x13 in Object.wait() [df87f000..df87fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90fdd0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90fdd0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '8' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x6cfb70 nid=0x12 in Object.wait() [df97f000..df97fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90fe50> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90fe50> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '7' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x612510 nid=0x11 in Object.wait() [dfa7f000..dfa7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90fed0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90fed0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '6' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x2ad4b0 nid=0x10 in Object.wait() [dfb7f000..dfb7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90ff50> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90ff50> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '5' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x2acf18 nid=0xf in Object.wait() [e057f000..e057fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea90ffd0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea90ffd0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '4' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x313100 nid=0xe in Object.wait() [e067f000..e067fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea910050> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea910050> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '3' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x312f68 nid=0xd in Object.wait() [e077f000..e077fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea9100d0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea9100d0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '2' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x312dd0 nid=0xc in Object.wait() [e247f000..e247fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea910150> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea910150> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '1' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x34fa58 nid=0xb in Object.wait() [e257f000..e257fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea9101d0> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea9101d0> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"ExecuteThread: '0' for queue: 'weblogic.kernel.Default'" daemon prio=5 tid=0x9e3bc0 nid=0xa in Object.wait() [e267f000..e267fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea910250> (a weblogic.kernel.ExecuteThread)
at java.lang.Object.wait(Object.java:426)
at weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:126)
- locked <ea910250> (a weblogic.kernel.ExecuteThread)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:145)
"Signal Dispatcher" daemon prio=10 tid=0xdfc70 nid=0x7 waiting on condition [0..0]
"Finalizer" daemon prio=8 tid=0xd9158 nid=0x4 in Object.wait() [fa97f000..fa97fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea3e3398> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:111)
- locked <ea3e3398> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:127)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=10 tid=0xd8f18 nid=0x3 in Object.wait() [faa7f000..faa7fc24]
at java.lang.Object.wait(Native Method)
- waiting on <ea3e3400> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:426)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:113)
- locked <ea3e3400> (a java.lang.ref.Reference$Lock)
"main" prio=5 tid=0x2d0c0 nid=0x1 in Object.wait() [ffbfe000..ffbfe624]
at java.lang.Object.wait(Native Method)
- waiting on <ea3e3430> (a weblogic.t3.srvr.T3Srvr)
at java.lang.Object.wait(Object.java:426)
at weblogic.t3.srvr.T3Srvr.waitForDeath(T3Srvr.java:1135)
- locked <ea3e3430> (a weblogic.t3.srvr.T3Srvr)
at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:379)
at weblogic.Server.main(Server.java:32)
"VM Thread" prio=5 tid=0xd8898 nid=0x2 runnable
"VM Periodic Task Thread" prio=10 tid=0xde9d0 nid=0x5 waiting on condition
"Suspend Checker Thread" prio=10 tid=0xdf320 nid=0x6 runnable
No comments:
Post a Comment