biolockj

Class Pipeline

java.lang.Object

biolockj.Pipeline

public class Pipeline
extends Object

This class initializes and executes each BioModule configured for execution.
BioModules that generate scripts are monitored until all scripts are complete, before moving on to the next module.

Method Summary

Modifier and Type	Method and Description
protected static void	deleteIncompleteModule(BioModule module) Delete and recreate incomplete module directory.
static void	executeModule(BioModule module) Execute a single pipeline module.
protected static void	executeModules() This method executes all new and incomplete modules Before/after a module is executed, set persistent module status by creating status indicator files.
protected static File	getMetadata(BioModule bioModule) If the bioModule is complete and contains a metadata file in its output directory, return the metadata file, since it must be a new version.
static List<BioModule>	getModules() Return a list of BioModules constructed by the BioModuleFactory
static String	getStatus() Return if no pipelineException has been thrown, otherwise return
protected static boolean	initializeModules() Initialization occurs by calling BioModule methods on configured modules Create module sub-directories under "internal.pipelineDir" as ordered in Config file. Reset the SummaryUtil module so previous summary descriptions can be used for completed modules Delete incomplete module contents if restarting a failed pipeline "output" directory Call refreshOutputMetadata(BioModule) to cache metadata if output by a complete module Call refreshRCacheIfNeeded(BioModule) to cache R fields after 1st R module runs Verify dependencies with BioModule.checkDependencies()
static void	initializePipeline() This method initializes the Pipeline by building the modules and checking module dependencies.
protected static boolean	poll(ScriptModule module) The ScriptModule.getScriptDir() will contain one main script and one ore more worker scripts. An empty file with appended to the script name is created when execution begins. If successful, an empty file with appended to the script name is created. Upon failure, an empty file with appended to the script name is created. Script status is polled each minute, determining status by counting indicator files. Log outputs the # of started, failed, and successful scripts (if any change). Log repeats the previous message every 10 minutes if no status change is detected.
protected static void	refreshOutputMetadata(BioModule module) Call MetaUtil to refresh the metadata cache if a new metadata file was output by the bioModule .
protected static void	refreshRCacheIfNeeded(BioModule module) Refresh R cache if about to run the 1st R module.
static void	runDirectModule(Integer moduleID) If moduleName is null, run all modules, otherwise only run the specified module.
static void	startPipeline() This method initializes and executes the BioModules set in the BioLockJ configuration file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

executeModule

public static void executeModule(BioModule module)
                          throws Exception

Execute a single pipeline module.

Parameters:

module - BioModule current module

Throws:

Exception - if runtime errors occur

getModules

public static List<BioModule> getModules()

Return a list of BioModules constructed by the BioModuleFactory

Returns:

List of BioModules

getStatus

public static String getStatus()

Return if no pipelineException has been thrown, otherwise return

Returns:

pipeline status (success or failed)

initializePipeline

public static void initializePipeline()
                               throws Exception

This method initializes the Pipeline by building the modules and checking module dependencies.

Throws:

Exception - if errors occur

runDirectModule

public static void runDirectModule(Integer moduleID)
                            throws Exception

If moduleName is null, run all modules, otherwise only run the specified module.

Parameters:

moduleID - of a single module to run

Throws:

Exception - if any fatal error occurs during execution

startPipeline

public static void startPipeline()
                          throws Exception

This method initializes and executes the BioModules set in the BioLockJ configuration file.

Throws:

Exception - if any fatal error occurs during execution

deleteIncompleteModule

protected static void deleteIncompleteModule(BioModule module)
                                      throws Exception

Delete and recreate incomplete module directory.

Parameters:

module - BioModule

Throws:

Exception - if errors occur

executeModules

protected static void executeModules()
                              throws Exception

This method executes all new and incomplete modules
Before/after a module is executed, set persistent module status by creating status indicator files. Incomplete modules have an empty file in the module directory.
Complete modules have an empty file in the module directory.
BioModules are run in the order listed in the Config file.

Execution steps:

1. File is added to the module directory
2. Execute refreshOutputMetadata(BioModule) to cache updated metadata, if generatede.
3. Run module scripts, if any, polling 1/minute for status until all scripts complete or time out.
4. File is replaced by as status indicator

Throws:

Exception - if script errors occur

getMetadata

protected static File getMetadata(BioModule bioModule)

If the bioModule is complete and contains a metadata file in its output directory, return the metadata file, since it must be a new version.

Parameters:

bioModule - BioModule

Returns:

New metadata file (or null)

initializeModules

protected static boolean initializeModules()
                                    throws Exception

Initialization occurs by calling BioModule methods on configured modules

1. Create module sub-directories under "internal.pipelineDir" as ordered in Config file.
   
2. Reset the SummaryUtil module so previous summary descriptions can be used for completed modules
3. Delete incomplete module contents if restarting a failed pipeline "output" directory
   
4. Call refreshOutputMetadata(BioModule) to cache metadata if output by a complete module
   
5. Call refreshRCacheIfNeeded(BioModule) to cache R fields after 1st R module runs
   
6. Verify dependencies with BioModule.checkDependencies()
   

Returns:

true if no errors are thrown

Throws:

Exception - thrown if propagated by called methods

poll

protected static boolean poll(ScriptModule module)
                       throws Exception

The ScriptModule.getScriptDir() will contain one main script and one ore more worker scripts.
An empty file with appended to the script name is created when execution begins.
If successful, an empty file with appended to the script name is created.
Upon failure, an empty file with appended to the script name is created.
Script status is polled each minute, determining status by counting indicator files.
Log outputs the # of started, failed, and successful scripts (if any change).
Log repeats the previous message every 10 minutes if no status change is detected.

Parameters:

module - ScriptModule

Returns:

true if all scripts are complete, regardless of status

Throws:

Exception - thrown to end pipeline execution

refreshOutputMetadata

protected static void refreshOutputMetadata(BioModule module)
                                     throws Exception

Call MetaUtil to refresh the metadata cache if a new metadata file was output by the bioModule .

Parameters:

module - BioModule

Throws:

Exception - if unable to refresh the cache

refreshRCacheIfNeeded

protected static void refreshRCacheIfNeeded(BioModule module)
                                     throws Exception

Refresh R cache if about to run the 1st R module.

Parameters:

module - BioModule

Throws:

Exception - if errors occur