Molecule deletion; bad SQL grammar

User 677b9c22ff

01-10-2008 17:40:50

Hi,


I had a matrix of 200k molecules and tried to delete 70k molecules.


After some seconds I got the following error, which I also attached as a text file because its pretty large.





What is the workaround? Deleting fewer than 32k substances (8 bit) or


exporting the good hits (which can be bad if its a large DB).





Bye


Tobias





Code:



org.springframework.jdbc.BadSqlGrammarException: Molecule deletion; bad SQL grammar [


 WHERE (C6H12O6.CD_ID IN (


2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,78,79,80,81,82,83,85,86,87,88,89,90,91,92,93,94,95,97,98,





...snip





 nested exception is java.sql.SQLException: Statement too complex. Try rewriting the query to remove complexity. Eliminating many duplicate expressions or breaking up the query and storing interim results in a temporary table can often help resolve this error.


Caused by: java.sql.SQLException: Statement too complex. Try rewriting the query to remove complexity. Eliminating many duplicate expressions or breaking up the query and storing interim results in a temporary table can often help resolve this error.


   at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)


   at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)


   at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)


   at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)


   at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)


   at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)


   at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)


   at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)


   at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)


   at chemaxon.jchem.db.UpdateHandler.deleteRows(UpdateHandler.java:2724)


   at chemaxon.jchem.db.UpdateHandler.deleteRows(UpdateHandler.java:2622)


   at com.im.df.impl.db.dao.JChemBaseDao.deleteIds(JChemBaseDao.java:1125)


   at com.im.df.impl.db.DBSchemaManager$6.doInTransaction(DBSchemaManager.java:994)


   at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:127)


   at com.im.df.impl.db.DBSchemaManager.deleteIds(DBSchemaManager.java:991)


   at com.im.df.impl.db.DBEntityDataProviderImpl.deleteIdsImpl(DBEntityDataProviderImpl.java:208)


   at com.im.df.impl.db.DBEntityDataProviderImpl.deleteIds(DBEntityDataProviderImpl.java:193)


   at com.im.ijc.core.api.actions.DeleteRowsAction$2.phase1InRequestProcessor(DeleteRowsAction.java:135)


   at com.im.df.util.BackgroundRunner.processTheTask(BackgroundRunner.java:112)


   at com.im.df.util.BackgroundRunner$1.run(BackgroundRunner.java:79)


   at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:561)


   at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:986)


Caused by: java.sql.SQLException: Java linkage error thrown during load of generated class org.apache.derby.exe.acdba306d9x011cxb419x6dadx00000f1bfb704.


   at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)


   at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)


   at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)


   at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)


   ... 19 more


Caused by: java.sql.SQLException: Java exception: 'constant_pool(72470 > 65535): java.io.IOException'.


   at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)


   at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)


   at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)


   at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)


   ... 20 more


Caused by: java.io.IOException: constant_pool(72470 > 65535)


   at org.apache.derby.iapi.services.classfile.ClassFormatOutput.limit(Unknown Source)


   at org.apache.derby.iapi.services.classfile.ClassFormatOutput.putU2(Unknown Source)


   at org.apache.derby.iapi.services.classfile.ClassHolder.put(Unknown Source)


   at org.apache.derby.iapi.services.classfile.ClassHolder.getFileFormat(Unknown Source)


   at org.apache.derby.impl.services.bytecode.BCClass.getClassBytecode(Unknown Source)


   at org.apache.derby.impl.services.bytecode.GClass.getGeneratedClass(Unknown Source)


   at org.apache.derby.impl.sql.compile.ExpressionClassBuilder.getGeneratedClass(Unknown Source)


   at org.apache.derby.impl.sql.compile.StatementNode.generate(Unknown Source)


   at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)


   at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)


   at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown Source)


   ... 15 more








ChemAxon fa971619eb

01-10-2008 19:06:00

I'm not surprised that there are problems deleting such a huge number of rows. I can't actually reproduce that error, but do get out of memory errors.





The workaround is certainly to do the deletion in smaller chunks. 10,000 seems to be safe.





We will look into alternative approaches to deleting larger numbers.





Tim