REST web services performance

User 773d472e7f

23-04-2014 15:38:11


WS2012 (64bit) | MySQL | JChem 6.2.1 | JChem REST Webservices | 16GB RAM






21 Million distinct molecules structure table 






We have observed a big performance difference between accessing the jchem web services from a web browser compared to accessing the web services via the Microsoft WCS service (running on IIS 7.5). In both cases the tests were performed on the same server as the jchem/webservice server




 




substructure search: 




WCS : typical result in 8 seconds.




Browser : same search result takes several minutes.




 




Please would you advise why this is the case and how we can obtain the same performance via the browser interface.


 


Many thanks


 


Bernard


ChemAxon 13811e1703

23-04-2014 16:32:34

Hi Bernard,


Can you send us the whole request json? Is it exactly the same for both cases?


Regards,
Peter 

User 773d472e7f

29-04-2014 07:27:44

Hi Peter


Apologies with delay with this reply. We have continued to test and have not been able to reproduce results consistently. We have found that the JChem web services searching is slow and  unstable.


I have installed Instant JChem (IJC) on this server and connected it to the same JChem database.


In this case IJC returns substructure search results within a mattar of seconds.


This help problem solve this issue?


is it possible to make IJC use the same cache as the REST webservices cache?


regards


Bernard


 


 

ChemAxon 13811e1703

29-04-2014 07:48:42

Hi Bernard,

It is not possible, IJC is a desktop application which is using the client's resources for caching and calculations. Can you send us a sample request which looks unstable for you?

Thanks,
Peter 

User 773d472e7f

29-04-2014 08:46:35

Hi Peter,


here is the json query request that we have used, it returns 8 hits from the 21 million molecule table:


{
    "searchOptions": {
"queryStructure": "COc1cc(NC(=O)c2ccc(=O)[nH]c2)cc(OC)c1",
"searchType": "SUBSTRUCTURE"
},
"paging": {
"offset": 0,
"limit": 10
},
"display": {
"include": ["cd_id", "akos_number"]
}
}

After the cache is built (40 minutes) This returns a result after about two minutes.  However after about 4-8 consecutive searches the service returns out of memory error:


{"errorCode":0,"errorMessage":"Java heap space","rootCause":"java.lang.OutOfMemoryError"}


If I use IJC (also installed on the server.) the result (for the same molecule as above) is returned within 3 seconds after the cache is built (40 minutes).


 


Regards


 


Bernard

User 773d472e7f

06-05-2014 16:39:56

We need urgently an answer. This problem hinders us to release the SSS capabilities to AKosSamples which is a serious business project for us.


With best regards,


Alex

ChemAxon 13811e1703

07-05-2014 09:07:33

Hi Alex,

In case of  
OutOfMemoryError I would recommend increasing the maximum memory allocation parameter of the JVM through the -Xmx option, here are some hints based on the size of the structure table(s):
http://www.chemaxon.com/jchem/doc/admin/Performance.html#hwrequirements />
(Same amount of memory should be enough for the JChem Web Services as the desktop application, are you running big number of searches parallel or what could be the difference?)

Regards,
Peter 

User 773d472e7f

07-05-2014 09:29:24

Hi Peter,


Please understand our question:


Using the same memory allocation (9GB) we get  very different results betwen IJC (running on the same server) and the web services, WHY?


IJC gives result in seconds: Web services gives results after many minutes and then soon runs out of memory.


What does this mean?


incorrect installation of web services?


specific configuration of webservices required?


Are there use cases similar to ours that actually work out there! 


bug in web services? 


We have invested much time and effort in trying to get this to work for us but so far have no reliable result.


regards


Bernard

ChemAxon 13811e1703

07-05-2014 10:49:41

Hi Bernard,

There is no other specific configuration for this and I din't know about this problem on any other environment.

Can you send us the corresponding log files of the JChem Web Services from the server? (logs/chemaxon.log) 

Regards,
Peter 

User 773d472e7f

07-05-2014 14:50:24


  1. I assume you have customers with databases of ca. 20 million structures using the combination Oracle and JChem cartridge. What is an average time for a SSS?

  2. I have an Oracle installation, 20 million compounds and on the same machine IJChem. Do I assume correctly that one can create a so-called server cache. IJChem is using the server cache. Yes, no?

  3. I have an Oracle installation, 20 million compounds and on the same machine IJChem. Can one build a different cache, a so called private IJChem cache. Yes, how would you do it? No?

  4. It seems we are able to build a server cache and a private cache on our server that uses MySQL. Why is the speed difference so large?


You and we are alredy working on this questions. Bernard will send logfiles.



  1. If we use the REST service, we crash the server cash after a few SSS searches. Why? 


I asked this questions a long time ago. Is there any progress?



  1. Why can't one read the cash from a file. Why is it necessara to build the cash fresh each time, after it crashes.


There are also more different questions from Hans-Jürgen.


We appreciate your help.


Alex


Below is the configuration schem settings for IJChem.


User 773d472e7f

08-05-2014 07:56:11

Hi Peter,


 


Please find attached the set of logs from the REST web services at the time when I was running tests.


The catalina log looks interesting in that it refers to memory leaks ......


regards


Bernard

ChemAxon 13811e1703

08-05-2014 10:24:40

Hi Bernard and Alex,


Thanks for the log files and the detailed information!


If I understand well, you are trying to compare search performance on different databases (IJChem configuration is Oracle, JChem Web Services connected to MySQL). Is it true? Speed difference can be caused by many things, I cannot tell the exact cause right now.


How many CPU cores are visible to the JVM running the Web Service and how many for the Instant JChem application?


For better understanding of JChem cache please have a look at: http://www.chemaxon.com/jchem/doc/dev/dbconcepts/#structurecache


Instant JChem, JChem Web Services and JChem Cartridge all three of them are using the JChem Java libraries for searching in relational databases. The cache for searching is created and maintained by the JChem libraries per JVM. Shutting down (crashing) servers or restarting the Instant JChem application will destroy the cache, and it will be rebuilt on next search request. This cache cannot be persisted to file or database, it is stored in memory and cannot be shared among JVM instances.


OutOfMemoryError on consecutive searches could be caused by a bug in JChem Web Services, and hopefully we can fix it in the next release.


If it is possible, please try this preview minor release of the JChem Web Services:


https://shared.chemaxon.com/projects/webservices/beta-release/webservices2-6.2.3-b1315.war


Feel free to contact me privately by email!


Regards,
Peter
plajko _at_ chemaxon.com

User 773d472e7f

08-05-2014 10:56:59

Hi Peter


Thanks for the reply. My apologies for the confusion: except for Alex's last entry (which perhaps belongs on a different enquiry)  this enquiry is about a single server with a single MySQL JChem Database:


WS2012 (64bit) | MySQL | JChem 6.2.1 | JChem REST Webservices | 16GB RAM


Just to clarify: the most perplexing finding is the difference in performance on the SAME server between JChem WebServices and IJC.


IJC = Fast :: WebServices = Slow :: This is a consistant finding.


As you state both of these systems are using the same JChem Java libraries for searching in relational databases. Clearly these libraries are capable of delivering the performance we require but so far only with IJC.


This implies either some fundamental design issue in the WebServices implementation or some configuration issue on our side ......


regards


Bernard


 




 


 

User 773d472e7f

08-05-2014 15:18:32

This might be confusing that Bernard and I ask questions in the same entry. However, we both try to get a thorough understanding of the cache technology and try understand performance that is dependent on the cache technology.


 



  1. I  assume you have customers with databases of ca. 20 million
    structures using the combination Oracle and JChem cartridge. What is an
    average time for a SSS? --- Please answer

  2. I have an Oracle installation, 20 million compounds and on the same
    machine IJChem. Do I assume correctly that one can create a so-called
    server cache. IJChem is using the server cache. Yes, no? --- You say there is only one type of cache. I can confirm. On the Oracle installtion is only one cache, and I can access this cache either via IJC or via a command line from SQLPlus. This means there is a possibility to access one cache with two different procedures. I don't quite understand what is a JVM instance. Is javaw.exe an JVM instance?You are saying. "This cache cannot be persisted to file or database, it is stored in memory and cannot be shared among JVM instances." However, it might be wise to allow to persist this cache!

  3. I have an Oracle installation, 20 million compounds and on the same
    machine IJChem. Can one build a different cache, a so called private
    IJChem cache. Yes, how would you do it? No? --- You say no!

  4. It seems we are able to build a server cache and a private cache on
    our server that uses MySQL. Why is the speed difference so large? --- It seems we are wrong. There is only one type of cache. However, the web service seems to do something for a very long time before it does the actual retrieval of the hits. It seems this is not the case if we use IJC? Maybe you could concentrate on finding out what causes this delay.


Thanks for your help.


Alex




 


 


 



ChemAxon 13811e1703

13-05-2014 17:38:40

Hi Alex and Bernard,

Average time for SSS is depending on wide range of conditions and parameters. Cannot tell you exactly comparable values..

Still we are investigating the issues raised by this forum thread, thanks for your participation and patience!

Regards,
Peter 

User e8902f42d3

11-08-2014 16:21:02

We need urgently an answer. This problem
hinders us to release the SSS capabilities to AKosSamples which is a
serious business project for us.


regards


ava


Clck Pencil

ChemAxon 13811e1703

13-08-2014 15:21:54

Hi Ava,

Can you please summarize the exact problem, and which version of the JChem Web Services is used?
Please try the latest release from our website!

Regards,
Peter