biolockj.module.report.otu

Class RarefyOtuCounts

java.lang.Object

biolockj.module.BioModuleImpl

biolockj.module.ScriptModuleImpl

biolockj.module.JavaModuleImpl

biolockj.module.report.otu.OtuCountModule

biolockj.module.report.otu.RarefyOtuCounts

All Implemented Interfaces:

BioModule, JavaModule, ScriptModule, Comparable<BioModule>

public class RarefyOtuCounts
extends OtuCountModule

This BioModule applies a mean iterative post-OTU classification rarefication algorithm so that each output sample will have approximately the same number of OTUs. Samples are rarefied to a configured quantile, most commonly (and most appropriately) to the default 0.50 quantile, otherwise known as the median. Users can configure this module to discard all samples with less OTUs than the rarefication threshold. Each sample is rarefied the configured number of times and the average OTU count is used for each sample. Note that even if a previous module removed singletons, this approach can yield new singleton OTU assignments but these are less likely to be due to contaminant and thus, should generally be allowed in the OTU table output.

GUI Module Name

Rarefy OTU Counts

Field Summary

Fields

Modifier and Type	Field and Description
protected static String	LOW_ABUNDANT_CUTOFF Config Posivite Double property to define minimum percentage of samples that must contain an OTU.
protected static String	NUM_ITERATIONS Config Positive Integer property "rarefyOtuCounts.iterations" defines the number of iterations to randomly select the "rarefyOtuCounts.quantile"% of OTUs.
protected static String	QUANTILE Config Positive Double property "rarefyOtuCounts.quantile" defines quantile for rarefication.
protected static String	REMOVE_LOW_ABUNDANT_SAMPLES Config Boolean property "rarefyOtuCounts.rmLowSamples" if TRUE, all samples below the quantile sample are removed.

Fields inherited from class biolockj.module.JavaModuleImpl

BLJ_OPTIONS

Fields inherited from class biolockj.module.BioModuleImpl

GZIP_EXT, LOG_EXT, PDF_EXT, RETURN, SH_EXT, TAB_DELIM, TSV_EXT, TXT_EXT

Fields inherited from interface biolockj.module.ScriptModule

SCRIPT_BATCH_SIZE, SCRIPT_DEFAULT_HEADER, SCRIPT_NUM_THREADS, SCRIPT_PERMISSIONS, SCRIPT_TIMEOUT

Fields inherited from interface biolockj.module.BioModule

MAIN_SCRIPT_PREFIX, OUTPUT_DIR, TEMP_DIR

Constructor Summary

Constructors

Constructor and Description
RarefyOtuCounts()

Method Summary

Modifier and Type	Method and Description
void	checkDependencies() Validate module dependencies: Require Config. exists Require Config. is positive integer Require Config. is positive integer Verify Config. is positive integer if set
void	cleanUp() Update ParserModuleImpl OTU_COUNT field name.
protected static void	generateOtuput(File file, TreeMap<String,Long> otuCounts) Print the output file wit rarefied counts.
protected static List<String>	getData(String sampleId, TreeMap<String,Long> otuCounts) Get OTU count data for the given sampleId.
protected List<String>	getLowAbundantSamples() Return a list of low abundant samples, as defined by "rarefyOtuCounts.lowAbundantCutoff"
protected Long	getNumOtusForQuantile(TreeMap<String,TreeMap<String,Long>> sampleOtuCounts) Get the quantile number of OTUs.
protected static List<String>	getRandomQuantileOtus(List<String> data, Long quantileNum) Select random OTUs based on the quantileNum from the list of OTUs in data.
String	getSummary() Produce summary message with min, max, mean, and median number of reads.
protected TreeMap<String,Long>	rarefy(String sampleId, TreeMap<String,Long> otuCounts, long quantileNum) Rarefy the data by taking the average value of "rarefyOtuCounts.iterations"
void	runModule() Apply the quantile Config to the number of OTUs per sample to calculate the maximum OTU count per sample.

Methods inherited from class biolockj.module.report.otu.OtuCountModule

getInputFiles, isOtuModule, isValidInputModule

Methods inherited from class biolockj.module.JavaModuleImpl

buildScript, executeTask, getSource, getWorkerScriptFunctions, markStatus, moduleComplete, moduleFailed

Methods inherited from class biolockj.module.ScriptModuleImpl

buildScriptForPairedReads, getJobParams, getMainScript, getRuntimeParams, getScriptDir, getScriptErrors, getTimeout, hasScripts

Methods inherited from class biolockj.module.BioModuleImpl

cacheInputFiles, compareTo, equals, findModuleInputFiles, getFileCache, getID, getModuleDir, getOutputDir, getPostRequisiteModules, getPreRequisiteModules, getTempDir, init, toString, validateFileNameUnique

Methods inherited from class java.lang.Object

clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface biolockj.module.ScriptModule

buildScriptForPairedReads, getJobParams, getMainScript, getScriptDir, getScriptErrors, getTimeout

Methods inherited from interface biolockj.module.BioModule

getID, getModuleDir, getOutputDir, getPostRequisiteModules, getPreRequisiteModules, getTempDir, init

Field Detail

LOW_ABUNDANT_CUTOFF

protected static final String LOW_ABUNDANT_CUTOFF

Config Posivite Double property to define minimum percentage of samples that must contain an OTU. Low abundance OTUs will be removed: "rarefyOtuCounts.lowAbundantCutoff"

See Also:

Constant Field Values

NUM_ITERATIONS

protected static final String NUM_ITERATIONS

Config Positive Integer property "rarefyOtuCounts.iterations" defines the number of iterations to randomly select the "rarefyOtuCounts.quantile"% of OTUs.

See Also:

Constant Field Values

QUANTILE

protected static final String QUANTILE

Config Positive Double property "rarefyOtuCounts.quantile" defines quantile for rarefication. The number of OTUs/sample are ordered, all samples with more OTUs than the quantile sample are subselected without replacement until they have the same number of OTUs as the quantile sample value. A quantile of 0.50 returns the median value.

See Also:

Constant Field Values

REMOVE_LOW_ABUNDANT_SAMPLES

protected static final String REMOVE_LOW_ABUNDANT_SAMPLES

Config Boolean property "rarefyOtuCounts.rmLowSamples" if TRUE, all samples below the quantile sample are removed.

See Also:

Constant Field Values

Constructor Detail

RarefyOtuCounts

public RarefyOtuCounts()

Method Detail

checkDependencies

public void checkDependencies()
                       throws Exception

Description copied from class: ScriptModuleImpl

Validate module dependencies:

• Require Config. exists
• Require Config. is positive integer
• Require Config. is positive integer
• Verify Config. is positive integer if set

Specified by:

checkDependencies in interface BioModule

Overrides:

checkDependencies in class ScriptModuleImpl

Throws:

Exception - thrown if missing or invalid dependencies are found

cleanUp

public void cleanUp()
             throws Exception

Update ParserModuleImpl OTU_COUNT field name.

Specified by:

cleanUp in interface BioModule

Overrides:

cleanUp in class BioModuleImpl

Throws:

Exception - thrown if any runtime error occurs

getSummary

public String getSummary()
                  throws Exception

Produce summary message with min, max, mean, and median number of reads.

Specified by:

getSummary in interface BioModule

Overrides:

getSummary in class ScriptModuleImpl

Returns:

Summary of BioModule execution

Throws:

Exception - if any error occurs

runModule

public void runModule()
               throws Exception

Apply the quantile Config to the number of OTUs per sample to calculate the maximum OTU count per sample. For each sample rarefy the configured number of times and output a file with the average counts. Update the metadata to add the new OTU_COUNT column with the new OTU count per sample.

Specified by:

runModule in interface JavaModule

Specified by:

runModule in class JavaModuleImpl

Throws:

Exception - thrown if any runtime error occurs

getLowAbundantSamples

protected List<String> getLowAbundantSamples()
                                      throws Exception

Return a list of low abundant samples, as defined by "rarefyOtuCounts.lowAbundantCutoff"

Returns:

List of Sample IDs

Throws:

Exception - if errors occur

getNumOtusForQuantile

protected Long getNumOtusForQuantile(TreeMap<String,TreeMap<String,Long>> sampleOtuCounts)
                              throws Exception

Get the quantile number of OTUs. If quantile = 0.5 the median value is returned.

Parameters:

sampleOtuCounts - TreeMap(SampleId, TreeMap(OTU, count)) OTU counts for every sample

Returns:

quantile number of OTUs

Throws:

Exception - if errors occur

rarefy

protected TreeMap<String,Long> rarefy(String sampleId,
                                      TreeMap<String,Long> otuCounts,
                                      long quantileNum)
                               throws Exception

Rarefy the data by taking the average value of "rarefyOtuCounts.iterations"

Parameters:

sampleId - Sample ID

otuCounts - OTU counts

quantileNum - Maximum number

Returns:

TreeMap(OTU, count) of rarefied data

Throws:

Exception - if errors occur

generateOtuput

protected static void generateOtuput(File file,
                                     TreeMap<String,Long> otuCounts)
                              throws Exception

Print the output file wit rarefied counts.

Parameters:

file - Output file

otuCounts - TreeMap(OTU, count)

Throws:

Exception - if errors occur

getData

protected static List<String> getData(String sampleId,
                                      TreeMap<String,Long> otuCounts)
                               throws Exception

Get OTU count data for the given sampleId.

Parameters:

sampleId - Sample ID

otuCounts - All OTU counts

Returns:

List of OTUs for the given sampleId

Throws:

Exception - if errors occur

getRandomQuantileOtus

protected static List<String> getRandomQuantileOtus(List<String> data,
                                                    Long quantileNum)

Select random OTUs based on the quantileNum from the list of OTUs in data.

Parameters:

data - List( OTUs )

quantileNum - Qunatile (range: 0.0 - 1.0)

Returns:

List of randomly selected OTUs up to the given quantile