tomcat falls over

My process using cartridge functions chugs along for so long and then it appears that tomcat falls over and thus jchemstreams is unavailable, stopping my process - any comments on how this situation could arise would be helpful.





ORA-29902: error in executing ODCIIndexStart() routine


ORA-29532: Java call


terminated by uncaught Java exception: java.lang.Exception: All registered JChem


service endpoints are unusable. Check if Tomcat is running


ORA-29902: error in executing ODCIIndexStart() routine


ORA-29532: Java call


terminated by uncaught Java exception: java.lang.Exception: All registered JChem


service endpoints are unusable. Check if Tomcat is running


ORA-20000: ORU-10028: line length overflow, limit of 255 chars per line

Please, could you specify the JChem, Oracle and operating system versions you are using?





Please, could you post a couple of dozens of lines from the final portions of <tomcat-home>/logs/catalina.out.

Hi Peter,





The version information is:


Oracle 9.2.0.1.0.


JChem 3.1.5


Tomcat itself is running on Linux Advanced version 2.1


Tomcat version 5.0.28.





It seems the same problem has not arisen since and the process seems to be happily working away for now. So shall we leave it for the time being and assume the cup is half full? If this happens again I will send you the relevant parts of the catalina.out.





Cheers,


Daniel.

Hi Daniel,





I recommend to consider upgrading to the newest Oracle 9i patch set (which appears to be 9.2.0.7) -- especially if your Oracle is running on Windows.





Cheers,


Peter

Can anybody help with the error message below from tomcat.


I cannot appear to run shutdown.sh without this occuring?


It is not clear to me why this error has started to occur (I am connected as the tomcat owner) Thanks for any comments...





Catalina.stop: java.net.ConnectException: Connection refused


java.net.ConnectException: Connection refused


at java.net.PlainSocketImpl.socketConnect(Native Method)


at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:305)


at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:171)


at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:158)


at java.net.Socket.connect(Socket.java:452)


at java.net.Socket.connect(Socket.java:402)


at java.net.Socket.<init>(Socket.java:309)


at java.net.Socket.<init>(Socket.java:124)


at org.apache.catalina.startup.Catalina.stopServer(Catalina.java:410)


at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)


at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)


at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)


at java.lang.reflect.Method.invoke(Method.java:324)


at org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:336)


at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:427)

shutdown.sh outputs this error message typically when Tomcat is not running. In rare cases, you can also see this error message, if Tomcat is running and the "shutdown port" has become blocked for some reason.





If Tomcat is running, you can terminate it by brute force on Linux with a command similar to the following:

Code:

ps -ef | grep tomcat | awk ' /java/ { system("kill " $2); } '

Code:

ps -ef | grep catalina | awk ' /java/ { system("kill -9 " $2); } '

is probably better. 'grep tomcat' works for me because, my Tomcat is started in the .../tomcat/... directory, so the directory appears in the command parameters. If your tomcat home directory is different (e.g. 'Tomcat'), you should use another string for selector (e.g. 'Tomcat'). 'grep catalina' is better, because 'catalina' appears in org.apache.catalina.startup.Bootstrap (the name of the class used internally by the Tomcat starter script). The safest way is to do a 'ps -ef' first and find the right process with "manual pattern recognition".

I think your second option of blocked/locked has happened (Not sure how)


After a re-boot of the machine that the tomcat process was on the problem persists (cannot shutdown but it is not available) and presumably the associated process was destroyed (thank for the syntax below, I presume that there are no issues with killing tomcat once caught?)





Do you know of a command that can be used to unblock a specified port number?





My other alternative is to switch port numbers and re-install the cartridge -no problem, but I'm concerned as you might expect that this could happen again through "normal" operations, also not sure what the status of the allocated memory is associated with this "purgatory" process.





Any comments gratefully received,


Daniel.

Please, could you post <tomcat-home>/logs/catalina.out?





BTW, I corrected the commands to kill Tomcat in my previous posts. (You have to use 'kill', of course, instead of 'echo'.)





Peter

Please find catalina.out attached (compressed) - I hope it is not too large and the relevant stuff is probably towards the EOF.





Please note \var\lock\subsys\Tomcatx does not exist.


(which was some advice I had read on another forum)

OK so I managed to wipe startup.sh from existence and then I went into a flat spin!!! If I was a horse they would have shot me by now.

So, even though I created my own problem with a blunder


in wiping startup.sh, it appears that tomcat can just fall over without any futher intervention from me and thus all query via tomcat subsequently go into "slow mode" as a result...





Can anyone comment as to why tomcat should fold?


Can I issue a programmatic check from Java that tomcat is not available?


I know can call .sh script from Java and thus PL/SQL.





Here is some error from catalina.out - is this to be expected?





SEVERE: Exception loading sessions from persistent storage


java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: chemaxon.sss.screen.CombinedFingerprint


at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1278)


at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1845)


at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1769)


at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1646)


at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1274)


at java.io.ObjectInputStream.readObject(ObjectInputStream.java:324)


at org.apache.catalina.session.StandardSession.readObject(StandardSession.java:1342)


at org.apache.catalina.session.StandardSession.readObjectData(StandardSession.java:885)


at org.apache.catalina.session.StandardManager.doLoad(StandardManager.java:416)


at org.apache.catalina.session.StandardManager.load(StandardManager.java:343)


at org.apache.catalina.session.StandardManager.start(StandardManager.java:657)


at org.apache.catalina.core.ContainerBase.setManager(ContainerBase.java:499)


at org.apache.catalina.startup.ContextConfig.managerConfig(ContextConfig.java:315)


at org.apache.catalina.startup.ContextConfig.start(ContextConfig.java:635)


at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:216)


at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)


at org.apache.catalina.core.StandardContext.start(StandardContext.java:4290)


at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1083)


at org.apache.catalina.core.StandardHost.start(StandardHost.java:789)


at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1083)


at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:478)


at org.apache.catalina.core.StandardService.start(StandardService.java:480)


at org.apache.catalina.core.StandardServer.start(StandardServer.java:2313)


at org.apache.catalina.startup.Catalina.start(Catalina.java:556)


at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)


at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)


at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)


at java.lang.reflect.Method.invoke(Method.java:324)


at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:287)


at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:425)


Caused by: java.io.NotSerializableException: chemaxon.sss.screen.CombinedFingerprint


at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1054)


at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1332)


at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1304)


at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1247)


at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1052)


at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:278)


at org.apache.catalina.session.StandardSession.writeObject(StandardSession.java:1415)


at org.apache.catalina.session.StandardSession.writeObjectData(StandardSession.java:902)

Quote:

Can I issue a programmatic check from Java that tomcat is not available?

From within PL/SQL, you can use the following statement for testing:

Code:

select jchem_core_pkg.getenvironment() from dual;

This statement will raise an exception, if Tomcat is not available.

Quote:

Here is some error from catalina.out - is this to be expected?

Shutdown Tomcat, purge the content of the <tomcat-home>/work directory, restart Tomcat.


I also suggest to try turning off Tomcat's http session persisting feature (unless you have a compelling reason for having it on):


http://forum.java.sun.com/thread.jspa?threadID=575302&messageID=2871584





Peter