biolockj.module.implicit.parser.r16s

Class QiimeParser

java.lang.Object

biolockj.module.BioModuleImpl

biolockj.module.ScriptModuleImpl

biolockj.module.JavaModuleImpl

biolockj.module.implicit.parser.ParserModuleImpl

biolockj.module.implicit.parser.r16s.QiimeParser

All Implemented Interfaces:

BioModule, ParserModule, JavaModule, ScriptModule, Comparable<BioModule>

public class QiimeParser
extends ParserModuleImpl

This BioModules parses QiimeClassifier output reports to build standard OTU abundance tables.

GUI Module Name

Qiime Parser

Field Summary

Fields

Modifier and Type	Field and Description
protected static List<String>	orderedQiimeIDs Qiime IDs are listed in the same order in each taxonomy level report.
protected static Map<String,String>	qiimeIdToSampleIdMap Convenience map, to convert Qiime ID to Sample ID
protected static Map<String,String>	sampleIdToQiimeIdMap Convenience map, to convert Sample ID to Qiime ID

Fields inherited from class biolockj.module.implicit.parser.ParserModuleImpl

NUM_OTUS

Fields inherited from class biolockj.module.JavaModuleImpl

BLJ_OPTIONS

Fields inherited from class biolockj.module.BioModuleImpl

GZIP_EXT, LOG_EXT, PDF_EXT, RETURN, SH_EXT, TAB_DELIM, TSV_EXT, TXT_EXT

Fields inherited from interface biolockj.module.ScriptModule

SCRIPT_BATCH_SIZE, SCRIPT_DEFAULT_HEADER, SCRIPT_NUM_THREADS, SCRIPT_PERMISSIONS, SCRIPT_TIMEOUT

Fields inherited from interface biolockj.module.BioModule

MAIN_SCRIPT_PREFIX, OUTPUT_DIR, TEMP_DIR

Constructor Summary

Constructors

Constructor and Description
QiimeParser()

Method Summary

Modifier and Type	Method and Description
void	cleanUp() By default, no cleanUp code is required.
protected int	getFileNameColumn(String line) Examines the header line to find the column index for "BioLockJFileName" which holds the demultiplexed sample file name.
List<File>	getInputFiles() Get only the lowest level report since it contains taxa info for all higher OTU reports.
protected static String	getOtuTableRowId(String id) This method is used to get an R-friendly Sample ID value by stripping out the quotes AND by replacing any Qiime ID found with the original Sample ID.
protected static String	getSampleId(String qiimeId) Get the Sample ID from qiimeIdToSampleIdMap.
protected static String	getSampleIdFromMappingFile(String qiimeId, int demuxIndex) The mapping file used the original sampleId when generating the demultipled files, as recorded in mapping file column - "BioLockJFileName".
protected void	initializeMaps() Sample IDs in the original metadata file may contain restricted characters that will be replaced with a "." via {link biolockj.module.implicit.qiime.BuildQiimeMapping}.
void	parseSamples() Parse lowest level QiimeClassifier report in the input directory.
void	runModule() To parse the taxonomy level reports output by QiimeClassifier: Call getInputFiles() Call initializeMaps() Call setOrderedQiimeIDs(File) Create ParsedSample for the OtuNodeImpl.getSampleId() if not yet created.
protected void	setOrderedQiimeIDs(File file) Qiime IDs are read in from the header line, converted to the sampleId, and saved to orderedQiimeIDs.

Methods inherited from class biolockj.module.implicit.parser.ParserModuleImpl

addOtuNode, buildOtuCountFiles, checkDependencies, getDepricatedOtuCountFields, getOtuCountField, getParsedSample, getSummary, isValid, setNumHitsFieldName, validateModuleOrder

Methods inherited from class biolockj.module.JavaModuleImpl

buildScript, executeTask, getSource, getWorkerScriptFunctions, isValidInputModule, markStatus, moduleComplete, moduleFailed

Methods inherited from class biolockj.module.ScriptModuleImpl

buildScriptForPairedReads, getJobParams, getMainScript, getRuntimeParams, getScriptDir, getScriptErrors, getTimeout, hasScripts

Methods inherited from class biolockj.module.BioModuleImpl

cacheInputFiles, compareTo, equals, findModuleInputFiles, getFileCache, getID, getModuleDir, getOutputDir, getPostRequisiteModules, getPreRequisiteModules, getTempDir, init, toString, validateFileNameUnique

Methods inherited from class java.lang.Object

clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface biolockj.module.JavaModule

moduleComplete, moduleFailed

Methods inherited from interface biolockj.module.ScriptModule

buildScript, buildScriptForPairedReads, getJobParams, getMainScript, getScriptDir, getScriptErrors, getTimeout, getWorkerScriptFunctions

Methods inherited from interface biolockj.module.BioModule

executeTask, getID, getModuleDir, getOutputDir, getPostRequisiteModules, getPreRequisiteModules, getTempDir, init, isValidInputModule

Field Detail

orderedQiimeIDs

protected static final List<String> orderedQiimeIDs

Qiime IDs are listed in the same order in each taxonomy level report. The values are cached here after being read from the first report.

qiimeIdToSampleIdMap

protected static final Map<String,String> qiimeIdToSampleIdMap

Convenience map, to convert Qiime ID to Sample ID

sampleIdToQiimeIdMap

protected static final Map<String,String> sampleIdToQiimeIdMap

Convenience map, to convert Sample ID to Qiime ID

Constructor Detail

QiimeParser

public QiimeParser()

Method Detail

cleanUp

public void cleanUp()
             throws Exception

Description copied from class: BioModuleImpl

By default, no cleanUp code is required.

Specified by:

cleanUp in interface BioModule

Overrides:

cleanUp in class BioModuleImpl

Throws:

Exception - thrown if any runtime error occurs

getInputFiles

public List<File> getInputFiles()
                         throws Exception

Get only the lowest level report since it contains taxa info for all higher OTU reports.

Specified by:

getInputFiles in interface BioModule

Overrides:

getInputFiles in class BioModuleImpl

Returns:

Input files

Throws:

Exception - if unable to obtain input files

parseSamples

public void parseSamples()
                  throws Exception

Parse lowest level QiimeClassifier report in the input directory. The classifier report lists 1 OTU/line with 1 column/sample, each column holds the OTU count for the QIIME ID in the column header. For each line, build a QiimeNode for each sample with a positive OTU count.

QiimeNodes will be created using QiimeID (not SampleID) in order to match the metadata file #SampleID

If ParserModuleImpl.isValid(OtuNode):

1. Create ParsedSample for the OtuNodeImpl.getSampleId() if not yet created.
2. Add the OtuNodeImpl.getCount() (1) to ParsedSample OTU count.

Specified by:

parseSamples in interface ParserModule

Specified by:

parseSamples in class ParserModuleImpl

Throws:

Exception - if error occurs while parsing classifier reports

runModule

public void runModule()
               throws Exception

To parse the taxonomy level reports output by QiimeClassifier:

1. Call getInputFiles()
2. Call initializeMaps()
3. Call setOrderedQiimeIDs(File)
4. Create ParsedSample for the OtuNodeImpl.getSampleId() if not yet created.
5. Add the OtuNodeImpl.getCount() (1) to ParsedSample OTU count.

Sample QIIME report line: (head otu_table_L2.txt):
# Constructed from biom file #OTU ID 3A.1 6A.1 120A.1 7A.1 k__Bacteria;p__Actinobacteria 419.0 26.0 90.0 70.0

Specified by:

runModule in interface JavaModule

Overrides:

runModule in class ParserModuleImpl

Throws:

Exception - thrown if any runtime error occurs

getFileNameColumn

protected int getFileNameColumn(String line)
                         throws Exception

Examines the header line to find the column index for "BioLockJFileName" which holds the demultiplexed sample file name.

Parameters:

line - Header line containing column names

Returns:

Index of Qiime mapping column "BioLockJFileName"

Throws:

Exception - If "BioLockJFileName" not found.

initializeMaps

protected void initializeMaps()
                       throws Exception

Sample IDs in the original metadata file may contain restricted characters that will be replaced with a "." via {link biolockj.module.implicit.qiime.BuildQiimeMapping}. The "BioLockJFileName" column will contain the original characters (and are used in the file name).

Throws:

Exception - if unable to initialize the maps

See Also:

http://qiime.org/scripts/validate_mapping_file.html

Convenience maps are initialized to simplify lookups:

• {@link #qiimeIdToSampleIdMap}
• {@link #sampleIdToQiimeIdMap}

setOrderedQiimeIDs

protected void setOrderedQiimeIDs(File file)
                           throws Exception

Qiime IDs are read in from the header line, converted to the sampleId, and saved to orderedQiimeIDs.

Parameters:

file - Taxonomy level report file

Throws:

Exception - If unable to read the file parameter

getOtuTableRowId

protected static String getOtuTableRowId(String id)
                                  throws Exception

This method is used to get an R-friendly Sample ID value by stripping out the quotes AND by replacing any Qiime ID found with the original Sample ID.

Parameters:

id - Sample ID

Returns:

formatted Sample ID

Throws:

Exception - If any QIIME ID does not have a corresponding Sample ID

getSampleId

protected static String getSampleId(String qiimeId)
                             throws Exception

Get the Sample ID from qiimeIdToSampleIdMap. Qiime removes special characters by output of BuildQiimeMapping, to create the qiimeId, which this method accepts as a parameter to lookup the Sample ID.

Parameters:

qiimeId - Qiime ID that was created from sample ID in a previous script

Returns:

sampleId Original sample ID from metadata file

Throws:

Exception - if the qiimeId is not found in qiimeIdToSampleIdMap

getSampleIdFromMappingFile

protected static String getSampleIdFromMappingFile(String qiimeId,
                                                   int demuxIndex)
                                            throws Exception

The mapping file used the original sampleId when generating the demultipled files, as recorded in mapping file column - "BioLockJFileName". Example: sampleId = gutSample_42 / qiimeId = gutSample.42 / "BioLockJFileName" = gutSample_42.fasta

Parameters:

qiimeId - Qiime corrected ID

demuxIndex - Qiime mapping column index for"BioLockJFileName"

Returns:

Original sampleId extracted from "BioLockJFileName"

Throws:

Exception - if unable to get sample ID from metadata file column "BioLockJFileName"