StackOverflowError when building an index on Extreg in ICJ

I am getting this error when building an index of the AKOS number (VARCHAR 1000) on the local version of a database with 250'000 structures.

RAM 8GB

Maximum Heap Space 3.64 GB

WIN8 64 bit system

 

org.springframework.transaction.TransactionSystemException: Could not commit JDBC transaction; nested exception is java.sql.SQLException: Java exception: ': java.lang.StackOverflowError'.
    at org.springframework.jdbc.datasource.DataSourceTransactionManager.doCommit(DataSourceTransactionManager.java:270)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754)
    at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:393)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:120)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at com.sun.proxy.$Proxy18.performOperation(Unknown Source)
    at com.im.df.impl.db.DBSchemaImpl.applyOperation(DBSchemaImpl.java:1101)
    at com.im.df.impl.db.DBIndexImpl.createIndex(DBIndexImpl.java:116)
    at com.im.ijc.core.nodes.NewIndexIterator$1.phase1InRequestProcessor(NewIndexIterator.java:118)
    at com.im.commons.progress.BackgroundRunner.processTheTask(BackgroundRunner.java:128)
    at com.im.commons.progress.BackgroundRunner$1.run(BackgroundRunner.java:91)
    at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1411)
    at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1991)
Caused by: java.sql.SQLException: Java exception: ': java.lang.StackOverflowError'.
    at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
    at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
    at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)
    at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
    at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
    at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
    at org.apache.derby.impl.jdbc.EmbedConnection.commit(Unknown Source)
    at org.apache.commons.dbcp.DelegatingConnection.commit(DelegatingConnection.java:334)
    at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.commit(PoolingDataSource.java:211)
    at org.apache.commons.dbcp.DelegatingConnection.commit(DelegatingConnection.java:334)
    at com.im.commons.db.datasource.BaseDataSource$CW.commit(BaseDataSource.java:621)
    at org.springframework.jdbc.datasource.DataSourceTransactionManager.doCommit(DataSourceTransactionManager.java:267)
    ... 14 more
Caused by: java.sql.SQLException: Java exception: ': java.lang.StackOverflowError'.
    at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
    at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
    ... 26 more
Caused by: java.lang.StackOverflowError
    at java.security.AccessController.doPrivileged(Native Method)
    at org.apache.derby.impl.store.raw.data.StreamFileContainer.privExists(Unknown Source)
    at org.apache.derby.impl.store.raw.data.StreamFileContainer.open(Unknown Source)
    at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openStreamContainer(Unknown Source)
    at org.apache.derby.impl.store.raw.xact.Xact.openStreamContainer(Unknown Source)
    at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.dropStreamContainer(Unknown Source)
    at org.apache.derby.impl.store.raw.xact.Xact.dropStreamContainer(Unknown Source)
    at org.apache.derby.impl.store.raw.data.DropOnCommit.update(Unknown Source)
    at java.util.Observable.notifyObservers(Unknown Source)
    at org.apache.derby.iapi.store.raw.xact.RawTransaction.notifyObservers(Unknown Source)
    at org.apache.derby.impl.store.raw.data.DropOnCommit.update(Unknown Source)
    at java.util.Observable.notifyObservers(Unknown Source)
    at org.apache.derby.iapi.store.raw.xact.RawTransaction.notifyObservers(Unknown Source)
    at org.apache.derby.impl.store.raw.data.DropOnCommit.update(Unknown Source)
    at java.util.Observable.notifyObservers(Unknown Source)
    at org.apache.derby.iapi.store.raw.xact.RawTransaction.notifyObservers(Unknown Source)
    at org.apache.derby.impl.store.raw.data.DropOnCommit.update(Unknown Source)
    at java.util.Observable.notifyObservers(Unknown Source)

I reduced the VARCHAR size to 13 and there was no problem to create an index. The index was build very fast. It took some time to copy the field.

Alex

Hi, this looks more like an error in Derby database engine itself more than the problem in IJC. However you may want to try to tune the -Xss java startup option to increase thread stack size. You can read about it at http://stackoverflow.com/questions/4967885/jvm-option-xss-what-does-it-do-exactly. The option can be set in instantjchem.conf file, wich is in etc folder of Instant JChem installation directory.

Anyway I'm glad that it gets resolved by decreasing the VARCHAR size.

Filip

I increased the value from 196 to 392. Let's see what happens with the 15 million structure db.

default_options="--branding instantjchem -J-Xms128m -J-XX:MaxPermSize=392m -J-Dderby.system.home=derby -J-Dnetbeans.logger.console=true -J-ea -J-Dorg.netbeans.ProxyClassLoader.level=1000 -J-DuseGtk=false -J-Dorg.netbeans.core.TimeableEventQueue.report=86400000 -J-Xmx4096m"
default_mac_options="${default_options} -J-Xmx1024m"

Alex

Reducing the VARCHAR size helped. The index was build in a few hours for 14 million records. The searches are now very fast.

We can close this thread with having a solution.

Thanks for the info.