biolockj.module.implicit

Class ImportMetadata

java.lang.Object

biolockj.module.BioModuleImpl

biolockj.module.implicit.ImportMetadata

All Implemented Interfaces:

BioModule, Comparable<BioModule>

public class ImportMetadata
extends BioModuleImpl

This BioModule validates the contents/format of the project metadata file and the related Config properties. If successful, the metadata is saved as a tab delimited text file.

GUI Module Name

Import metadata

Field Summary

Fields inherited from class biolockj.module.BioModuleImpl

GZIP_EXT, LOG_EXT, PDF_EXT, RETURN, SH_EXT, TAB_DELIM, TSV_EXT, TXT_EXT

Fields inherited from interface biolockj.module.BioModule

MAIN_SCRIPT_PREFIX, OUTPUT_DIR, TEMP_DIR

Constructor Summary

Constructors

Constructor and Description
ImportMetadata()

Method Summary

Modifier and Type	Method and Description
protected File	buildNewMetadataFile() Create a simple metadata file in the module output directory, with only the 1st column populated with Sample IDs.
void	checkDependencies() If restarting or running a direct pipeline execute the cleanup for completed modules.
void	cleanUp() Verify the metadata fields configured for R reports.
void	executeTask() If Config."metadata.filePath" is undefined, build a new metadata file with only 1 column of sample IDs.
protected String	formatMetaId(String sampleIdColumnName) Format the metadata ID to remove problematic invisible characters (particularly converted Excel files).
protected String	getQuotedValue(String val) The member variable quotedText caches the input held within a quoted block.
protected TreeSet<String>	getSampleIds() Extract the sample IDs from the file names with SeqUtil.getSampleId(String)
String	getSummary() The metadata file can be updated several times during pipeline execution.
protected boolean	inQuotes(String val) Method called each time a line from metadata contains the Config."metadata.columnDelim".
protected String	parseRow(String line, boolean isHeader) Method called to parse a row from the metadata file, where Config."metadata.columnDelim" separates columns.
protected void	verifyAllRowsMapToSeqFile(List<File> files) Verify every row (every Sample ID) maps to a sequence file
protected static void	verifyHeader(String cell, List<String> colNames, int colNum) Verify column headers are not null and unique

Methods inherited from class biolockj.module.BioModuleImpl

cacheInputFiles, compareTo, equals, findModuleInputFiles, getFileCache, getID, getInputFiles, getModuleDir, getOutputDir, getPostRequisiteModules, getPreRequisiteModules, getTempDir, init, isValidInputModule, toString, validateFileNameUnique

Methods inherited from class java.lang.Object

clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ImportMetadata

public ImportMetadata()

Method Detail

checkDependencies

public void checkDependencies()
                       throws Exception

Description copied from class: BioModuleImpl

If restarting or running a direct pipeline execute the cleanup for completed modules.

Specified by:

checkDependencies in interface BioModule

Specified by:

checkDependencies in class BioModuleImpl

Throws:

Exception - thrown if missing or invalid dependencies are found

cleanUp

public void cleanUp()
             throws Exception

Verify the metadata fields configured for R reports.

Specified by:

cleanUp in interface BioModule

Overrides:

cleanUp in class BioModuleImpl

Throws:

Exception - thrown if any runtime error occurs

executeTask

public void executeTask()
                 throws Exception

If Config."metadata.filePath" is undefined, build a new metadata file with only 1 column of sample IDs. Otherwise, import "metadata.filePath" file and call MetaUtil.refreshCache() to validate, format, and cache metadata as a tab delimited text file.

Specified by:

executeTask in interface BioModule

Specified by:

executeTask in class BioModuleImpl

Throws:

Exception - thrown if the module is unable to complete is task

getSummary

public String getSummary()
                  throws Exception

The metadata file can be updated several times during pipeline execution. Summary prints the file-path of the final metadata file, along with sample and field count (obtained from MetaUtil).

Specified by:

getSummary in interface BioModule

Overrides:

getSummary in class BioModuleImpl

Returns:

Module summary of metadata path, #samples and #fields

Throws:

Exception - if any error occurs

buildNewMetadataFile

protected File buildNewMetadataFile()
                             throws Exception

Create a simple metadata file in the module output directory, with only the 1st column populated with Sample IDs.

Returns:

Metadata file

Throws:

Exception - if unable to build the new file due to invalid params or I/O errors

formatMetaId

protected String formatMetaId(String sampleIdColumnName)

Format the metadata ID to remove problematic invisible characters (particularly converted Excel files). If the first Excel cell value starts with # symbol, Excel adds a ZERO WIDTH NO-BREAK space as an invisible character. Here we strip this value; See http://www.fileformat.info/info/unicode/char/feff/index.htm

Parameters:

sampleIdColumnName - Current name of metadata Sample ID column

Returns:

Formatted Sample ID column name

getQuotedValue

protected String getQuotedValue(String val)

The member variable quotedText caches the input held within a quoted block. This block can contain the Config."metadata.columnDelim" which will be read as a character. If val closes an open quoted block, the entire quotedBlock is returned (ending with Config."metadata.columnDelim" as all cells do) and the quotedText cache is cleared.

Parameters:

val - Parameter to evaluate

Returns:

Quoted text block (so far)

getSampleIds

protected TreeSet<String> getSampleIds()
                                throws Exception

Extract the sample IDs from the file names with SeqUtil.getSampleId(String)

Returns:

Ordered set of Sample IDs

Throws:

Exception - if any duplicate or invalid sample IDs are returned by SeqUtil

inQuotes

protected boolean inQuotes(String val)

Method called each time a line from metadata contains the Config."metadata.columnDelim". If the Config."metadata.columnDelim" is encountered within a quoted block, it should be interpreted as a character (not interpreted as a column delimiter).

Parameters:

val - Parameter to evaluate

Returns:

true if within open quotes

parseRow

protected String parseRow(String line,
                          boolean isHeader)
                   throws Exception

Method called to parse a row from the metadata file, where Config."metadata.columnDelim" separates columns. The quotedText member variable serves as a cache to build cell values contained in quotes which may include the Config."metadata.columnDelim" as a standard character. Each row increments rowNum member variable. When the header row is processed, colNames caches the field names.

Parameters:

line - read from metadata file

isHeader - is true for only the first row

Returns:

parsed row value

Throws:

Exception - if required Config values are missing or invalid

verifyAllRowsMapToSeqFile

protected void verifyAllRowsMapToSeqFile(List<File> files)
                                  throws Exception

Verify every row (every Sample ID) maps to a sequence file

Parameters:

files - List of sequence files

Throws:

ConfigViolationException - if unmapped Sample IDs are found

Exception - if other errors occur

verifyHeader

protected static void verifyHeader(String cell,
                                   List<String> colNames,
                                   int colNum)
                            throws Exception

Verify column headers are not null and unique

Parameters:

cell - value of the header column name

colNames - a list of column names read so far

colNum - included for reference in error message if needed

Throws:

Exception - if a column header is null or not unique