Hi Subbarao,
there is a tool in the Screen package that can be used to suggest reasonable threshold values.
This tool is a generic metric optimizer whose aim is to increase the enrichment ratio of virtual screening. Among many parameters that it can optimize threshold is also calculated.
The optimizer is available via a commandline tool called optimizemetrics. To run it you need three input files. First a random molecule set, for instance a subset of your target library you want to search for similar structures. Then you also need a set of structures that exhibit some sort of similarity. This can be structural or pharmacophoric or biological etc. You don't need too many of them, though 20 or so is a required minimum. Split these 20 structures in two independent subsets, the training set and the query set. Structures in the training set will be used as spikes, that is, they will be mixed in the random set. The aim of the optimization is to retrieve a given predefined percentage of these spikes.
Then decide which molecular descriptor you want to use, I suppose this is the ChemAxon's chemical fingerprint, and also the appropriate similarity/dissimilarity metric, I reckon this could be Tanimoto in your case.
It is recommended to use a hypothesis fingerprint during optimization, rather than individual fingerprints of the structures in the query set. You have three options here, Minimum, Median or Average. If your query set is not too diverse, use Minimum (this is often called consensus), otherwise I recommend Median.
Finally, you need to decide the spike retrieval percentage, by default this is 80%.
Bearing all the above considerations in mind, the command you need to execute is as follows:
optimizemetrics randomset.smiles trainingset.smiles queryset.smiles -k CF -c cfp.xml -M Tanimoto -t -H
In this commandline -k CF tells the optimizer to use chemical fingerprints for similarity calculation. -c cfp.xml defines the configuration file that specifies the parameters for the chemical fingerprint. You find a sample cfp.xml file in the examples/config folder in your JChem installation directory. This is an XML file that you can edit with a simple text editor. You may need to edit it to make sure that parameters are the same as in your similarity search.
-M Tanimoto specifies the metric, -t tells optimizer to set threshold only according to spikes percentage (80%, by default), and -H tells that hypothesis of the queryset has to be used. By default this is Minimum.
To play with some options, you can try this:
optimizemetrics randomset.smiles trainingset.smiles queryset.smiles -k CF -c cfp.xml -M Tanimoto -t -H Median -p 90
Here a median hypothesis is used and the spikes retrieval percentage is increased to 90%.
The above two command dump results (in XML) to the standard output. You need to look for a line like this (towards the end of output):
<ParametrizedMetric Name="queryset.smiles by Tanimoto" ActiveFamily="queryset.smiles" Metric="Tanimoto" Threshold="0.91">
The last value, Threshold is what you need. Be aware that this is a dissimilarity threshold, thus substract it from 1 to get a similiarity threshold.
I hope this helps. I know my answer is quite long but optimizemetrics is a fairly complex piece of software. Please get back to the forum if something is not discussed appropriately or in case you have further questions.
Regards,
Miklos