Tomcat memory allocation

Hello,





Just trying to get a feel for how much memory would be required for the cache of slightly more than 3M (3088758) records/molecules. A slightly smaller amount 3M seemed to load OK into less than 2Gb of memory however, it appears that no amount of memory that I allocate will hold this new number and the out of memory error occurs at the same point each time in the cache load. For instance I have 4Gb total memory available, I have cut right down the Oracle allocation and then startup tomcat using "startup -server -Xmx3000m" (which appears to be proportionally a lot more memory than the new amount of records added should require based on past experience) and I had hoped this would use all available memory, however this did not work and it stopped loading the cache at approximately the same place.





Any comments on how I can ensure that all available memory is being used by tomcat would be appreceiated.





Daniel.

Hi,





You shouldn't need that much memory for 3 million structures.


For typical drug-like structures you need around 100MB per 1 million molecule, so they should fit into 300MB.


You also need some memory for temporary allocations, but a total amount of around 400-500 MB should be fine.

Quote:

and then startup tomcat using "startup -server -Xmx3000m"

I don't think it works.


I don't know about you Tomcat version, but so far I haven't heard about Tomcat being able to parse JVM options like that.





Please see our advice in the documentation on how to set JVM options:





http://www.chemaxon.com/jchem/doc/admin/tomcat.html#env





Best regards,





Szilard

Hi Szilard,





Our exact situation is that previously just over 3 million loaded into the cache with 1600m allocated to tomcat...we added 30000 more records and larger cache sizes could not accomodate the new size up to the tomcat(?)/the physcial limit...we then decided to rollback the 30000, but now we find that the previous number which did load in 1600m now does not...You comment regarding approx 100Gb for 1 million small molecule means that we should be able to easily load our table - we used to be able to but not sure why we cannot now!? All in all not sure how to proceed...





Daniel.

Previously whilst the cache was loading the message "loading cache" appeared...now it appears for a short while and then disappears at the same point irrespective of the amount of memory previously allocated to tomcat? I'm not sure what has unintentionally changed in my configuration?

Well, the only thing that is apparently different now from when things worked is the JChem version itself.


In 3.1.5, all other things constant, we did not see this problem. In the latest test version "TEST_2006_05_17" the cache seems to stop loading (see previous forum message) some way through? Presumably you could confirm that you not see similar behaviour with this release?

I witness the same behaviour on a table with 1 million structures.

Could you please make this available from the download area...


jchemTEST_2006_04_19.tar.gz


(I don't think 3.1.6 has the exact stuff I need yet)

inhibox wrote:

I witness the same behaviour on a table with 1 million structures.

Dan,





Can you please check the documentation Szilard linked above about setting Tomcat parameters? I recite the relevant part from the manual:

Quote:

3.2 Setting JVM options for Tomcat * Windows running Tomcat 5 and later : Go to the "Apache Tomcat 5.0" folder in the Start Menu. Start the "Configure Tomcat application". Note: the configuration can only be started when Tomcat is not running. Select the "Java VM" tab in the configuration dialog. You will see some pre-defined lines in the "Java Options" test box. Append your Java options at the bottom of the option list. For example, to set server mode and allow 512MB of heap memory, append the following: -server -Xmx400m * Windows running Tomcat 4.1 or earlier : place the Java options into the CATALINA_OPTS environment variable. To do this, run Control Panel / System, select Environment Variables and create the CATALINA_OPTS variable, and set the desired option, for example "-server -Xmx400m". * Linux : place the Java options into the CATALINA_OPTS environment variable. For example: "-server -Xmx400m".

Szabolcs

OK - so I had been previously setting this but changed recently to supply the variables with the startup command for some reason - so this is definitely required to be set.





* Linux : place the Java options into the CATALINA_OPTS environment variable. For example: "-server -Xmx400m".





Thanks for your patience once again!


Daniel.