Cartridge memory error

Hi,

When I do substructure search over 70 million structures,  the jchem cartridge hangs with memory error messages.

 

 

I am using jchem cartridge 6.1.3 on CentOS 6.4 64-bit in PC (6-core cpu, 64GB memory) system.

The memory option is : jcserver.maxmem=40960m

I have stored the structure of ZINC and PubChem using JChemManager. The number of molecules is over 60 millions.

When I do substructure using SQLPLUS, I get the following error messages and the jchem server hangs. To redo the search, I have to reboot my linux system.

How can I treat this?

Regards,

 

 

 

 

 

 

 

 

 

select count(cd_id) from chemdb where JC_COMPARE( cd_structure,'c1cc(ccc1CNn2cnnc2)OCc3ccc(cc3)Cl', 't:s' ) = 1

*

ERROR at line 1:

ORA-29904: error in executing ODCIIndexClose() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: java.util.concurrent.ExecutionException:

chemaxon.jchem.db.DatabaseSearchException: java.lang.OutOfMemoryError: unable

to create new native thread

Caused by:

unable to create new native thread; nested exception is:

chemaxon.jchem.db.DatabaseSearchException: java.lang.OutOfMemoryError: unable

to create new native thread

Caused by:

unable to create new native thread; nested exception is:

java.lang.OutOfMemoryError: unable to create new native thread

ORA-06512: at "JCHEM.JCHEM_CORE_PKG", line 254

ORA-06512: at "JCHEM.JC_IDXTYPE_IM", line 601

ORA-29903: error in executing ODCIIndexFetch() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: chemaxon

 

 

select count(cd_id) from chemdb where JC_COMPARE( cd_structure,'COc1cc(ccc1OCc2ccc(cc2)Cl)CNn3cnnc3', 't:s' ) = 1

*

ERROR at line 1:

ORA-29904: error in executing ODCIIndexClose() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: java.util.concurrent.ExecutionException:

chemaxon.jchem.db.DatabaseSearchException: java.lang.OutOfMemoryError: unable

to create new native thread

Caused by:

unable to create new native thread; nested exception is:

chemaxon.jchem.db.DatabaseSearchException: java.lang.OutOfMemoryError: unable

to create new native thread

Caused by:

unable to create new native thread; nested exception is:

java.lang.OutOfMemoryError: unable to create new native thread

ORA-06512: at "JCHEM.JCHEM_CORE_PKG", line 254

ORA-06512: at "JCHEM.JC_IDXTYPE_IM", line 601

ORA-29903: error in executing ODCIIndexFetch() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: chemaxon

 

 

select count(cd_id) from chemdb where JC_COMPARE( cd_structure,'c1ccc(cc1)c2nn([n+](n2)c3ccc(cc3)[N+](=O)[O-])c4ccccc4', 't:s' ) = 1

*

ERROR at line 1:

ORA-29902: error in executing ODCIIndexStart() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: java.lang.OutOfMemoryError: unable to create new

native thread

ORA-06512: at "JCHEM.JCHEM_CLOB_PKG", line 81

ORA-06512: at "JCHEM.JC_IDXTYPE_IM", line 310

 

 

select count(cd_id) from chemdb where JC_COMPARE( cd_structure,'Cn1c(=O)n(c(=O)n(c1=O)Cc2ccc(c(c2)[N+](=O)[O-])OC)C', 't:s' ) = 1

*

ERROR at line 1:

ORA-29902: error in executing ODCIIndexStart() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: java.lang.OutOfMemoryError: unable to create new

native thread

ORA-06512: at "JCHEM.JCHEM_CLOB_PKG", line 81

ORA-06512: at "JCHEM.JC_IDXTYPE_IM", line 310

 

 

select count(cd_id) from chemdb where JC_COMPARE( cd_structure,'c1cc(ncc1Cl)NC=C(C#N)C#N', 't:s' ) = 1

*

ERROR at line 1:

ORA-29902: error in executing ODCIIndexStart() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: java.lang.OutOfMemoryError: unable to create new

native thread

ORA-06512: at "JCHEM.JCHEM_CLOB_PKG", line 81

ORA-06512: at "JCHEM.JC_IDXTYPE_IM", line 310

 

 

select count(cd_id) from chemdb where JC_COMPARE( cd_structure,'c1ccc2c(c1)C(=O)C(C2=O)(C3=C(CCCC3=O)[O-])O', 't:s' ) = 1

*

ERROR at line 1:

ORA-29902: error in executing ODCIIndexStart() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: java.lang.OutOfMemoryError: unable to create new

native thread

ORA-06512: at "JCHEM.JCHEM_CLOB_PKG", line 81

ORA-06512: at "JCHEM.JC_IDXTYPE_IM", line 310

 

 

select count(cd_id) from chemdb where JC_COMPARE( cd_structure,'c1cc(cc(c1)[N+](=O)[O-])NC(=O)CC#N', 't:s' ) = 1

*

ERROR at line 1:

ORA-29902: error in executing ODCIIndexStart() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: java.lang.OutOfMemoryError: unable to create new

native thread

ORA-06512: at "JCHEM.JCHEM_CLOB_PKG", line 81

ORA-06512: at "JCHEM.JC_IDXTYPE_IM", line 310

 

 

select count(cd_id) from chemdb where JC_COMPARE( cd_structure,'c1ccnc(c1)/C=C/2\C(=O)N=C(S2)N3CCCCC3', 't:s' ) = 1

*

ERROR at line 1:

ORA-29902: error in executing ODCIIndexStart() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: java.lang.OutOfMemoryError: unable to create new

native thread

ORA-06512: at "JCHEM.JCHEM_CLOB_PKG", line 81

ORA-06512: at "JCHEM.JC_IDXTYPE_IM", line 310

 

 

select count(cd_id) from chemdb where JC_COMPARE( cd_structure,'c1ccnc(c1)/C=C\2/C(=O)N=C(S2)N3CCCCC3', 't:s' ) = 1

*

ERROR at line 1:

ORA-29902: error in executing ODCIIndexStart() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: java.lang.OutOfMemoryError: unable to create new

native thread

ORA-06512: at "JCHEM.JCHEM_CLOB_PKG", line 81

ORA-06512: at "JCHEM.JC_IDXTYPE_IM", line 310

 

 

select count(cd_id) from chemdb where JC_COMPARE( cd_structure,'CCOc1ccc(cc1)N2CC(=O)SC2=S', 't:s' ) = 1

*

ERROR at line 1:

ORA-29902: error in executing ODCIIndexStart() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: java.lang.OutOfMemoryError: unable to create new

native thread

ORA-06512: at "JCHEM.JCHEM_CLOB_PKG", line 81

ORA-06512: at "JCHEM.JC_IDXTYPE_IM", line 310

 

 

select count(cd_id) from chemdb where JC_COMPARE( cd_structure,'Cc1cccc(n1)/C=N/NC(=O)CCc2ccccc2O', 't:s' ) = 1

*

ERROR at line 1:

ORA-29902: error in executing ODCIIndexStart() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: java.lang.OutOfMemoryError: unable to create new

native thread

ORA-06512: at "JCHEM.JCHEM_CLOB_PKG", line 81

ORA-06512: at "JCHEM.JC_IDXTYPE_IM", line 310

 

 

select count(cd_id) from chemdb where JC_COMPARE( cd_structure,'Cc1cccc(n1)/C=N/NC(=O)c2cccs2', 't:s' ) = 1

*

ERROR at line 1:

ORA-29902: error in executing ODCIIndexStart() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: java.lang.OutOfMemoryError: unable to create new

native thread

ORA-06512: at "JCHEM.JCHEM_CLOB_PKG", line 81

ORA-06512: at "JCHEM.JC_IDXTYPE_IM", line 310

 

 

select count(cd_id) from chemdb where JC_COMPARE( cd_structure,'Cc1cccc(n1)/C=N/NC(=O)CCc2ccc3c(c2)OCO3', 't:s' ) = 1

*

ERROR at line 1:

ORA-29902: error in executing ODCIIndexStart() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: java.lang.OutOfMemoryError: unable to create new

native thread

ORA-06512: at "JCHEM.JCHEM_CLOB_PKG", line 81

ORA-06512: at "JCHEM.JC_IDXTYPE_IM", line 310

 

 

select count(cd_id) from chemdb where JC_COMPARE( cd_structure,'Cc1cccc(n1)/C=N/NC(=O)OCc2ccc(cc2)OC', 't:s' ) = 1

*

ERROR at line 1:

ORA-29902: error in executing ODCIIndexStart() routine

ORA-29532: Java call terminated by uncaught Java exception:

java.rmi.ServerException: RemoteException occurred in server thread; nested

exception is:

java.rmi.RemoteException: java.lang.OutOfMemoryError: unable to create new

native thread

Hi,

It's strange for use, too, because we calculated that in normal circumstances this amount of memory should be enough.

Can you send use the average smiles length in the table:

select avg(length(cd_structure)) from chemdb;

And can you send us also the jcart.properties file from the cartridge/conf directory?

Thank you,

Tamas

Hi, Thank you for your interest.

Below, I attach the jcart .properties and other information.

Regards,

 

 

 

 

SQL> select count(*) from chemdb;

  COUNT(*)

----------

  58075147

SQL> select avg(length(cd_structure)) from chemdb;

AVG(LENGTH(CD_STRUCTURE))

-------------------------

               48.0196523

SQL>

 

 

zincdb:/home/chemaxon/jchem/cartridge/conf$cat jcart.properties

# Type of the descriptor used to reference the database service/instance (sid,

# service_name, url)

oracle.server.connectiontype=sid

 

# The name of the Oracle host

oracle.server.host=localhost

 

# The port where the Oracle listener accepts connections

oracle.server.port=1521

 

# Name of the Oracle database (sid)

oracle.server.instance=orcl

 

# The name of schema where JChem Cartridge has to be installed

jcart.owner.login=jchem

 

# DBA login is required to create the JChem Cartridge owner's schema, granting

# the JChem Cartridge owner the required system privileges, creating public

# synonyms and optionally creating a basic role for JChem Cartridge users. If

# you don't wish to specify a DBA login, please enter '\!'. The SQL statements

# to be executed with elevated privileges can be listed by executing the

# 'config-util.{sh,bat} list-dba-sqls' command in the jchem/cartridge

# directory.

dba.login=SYSTEM

 

# The host name of the JChem Server, as seen (pingable) from the Oracle host.

# In a single-host setup, it is "localhost" or 127.0.0.1.

jchem.server.host=localhost

 

# The port at which the JChem Server accepts connections

jchem.server.port=1099

 

# The maximum heap memory allocated to Java process in which the JChem Server

# is running. See

# http\://www.chemaxon.com/jchem/doc/admin/cartridge.html\#server_memory for

# details on how to calculate the required amount.

jcserver.maxmem=40960m

 

# Configuration file for logging

logging.config.file=conf/logging.properties

 

# Directory to store the log files

logging.output.dir=logs

 

# The structure cache id of the JChem Server instance currently being

# configured. The structure cache id must uniquely identify JChem structure

# caches across all JChem based application accessing the same structure

# tables.

structure.cache.id=JCC1

 

# The role for basic JChem Cartridge user privileges

basic.jcc.role=JCC_BASIC_ROLE

 

# Name of the JChem Server service.

service.name=JChemCartridgeService

 

oracle.connection.cache.ValidateConnection=true

sysprop.java.rmi.dgc.leaseValue=60000000

sysprop.sun.rmi.dgc.ackTimeout=60000000

sysprop.sun.rmi.dgc.server.gcInterval=60000000

sysprop.sun.rmi.transport.connectionTimeout=60000000

xx.opt.-OmitStackTraceInFastThrow=

Hi,

We found out that your problem is that the operating system can't create more processes during cache load. It is independent of the heap size setting. You can view your settings with the ulimit -a command, and then increase the maximum number of processes with the ulimit -u.

Regards,

Tamas

Hi, Tamas.

I greately appreciate your help. I will try the ulimit problem.

Thank you.

Regards,