Commit a68743c6 authored by Leo Gordon's avatar Leo Gordon
Browse files

tweak: documented the new command line options in Changelog

parent 63472d57
......@@ -15,26 +15,41 @@ Main highlights of the release:
NB: see these three features in action in the Long-multiplication pipelines
All the example pipelines have been grouped together under a new directory: Bio/EnsEMBL/Hive/Examples
(in beta) new configuration mechanism to -tweak parameters and attributes of pipelines either during pipeline initialization or afterwards.
For tweaking things during initialization we have extended init_pipeline.pl to understand -tweak commands.
For tweaking things after the pipeline database has been created there is a new tweak_pipeline.pl script that understands the same -tweaks :
-tweak 'pipeline.param[take_time]=20' # override a value of a pipeline-wide parameter; can also create an inexistent parameter
-tweak 'pipeline.hive_pipeline_name=new_name' # override a value of a hive_meta attribute
-tweak 'analysis[take_b_apart].param[base]=10' # override a value of an analysis-wide parameter; can also create an inexistent parameter
-tweak 'analysis[add_together].analysis_capacity=3' # override a value of an analysis attribute
-tweak 'analysis[add_together].batch_size=15' # override a value of an analysis_stats attribute
-tweak 'analysis[part_multiply].resource_class=urgent' # set the resource class of an analysis (whether a resource class with this name existed or not)
-tweak 'resource_class[urgent].LSF=-q yesteryear' # update or create a new resource description
In both cases you can print out the current value of things by using the '?' instead of the assignment:
-tweak 'pipeline.hive_pipeline_name?' # show the pipeline_name
-tweak 'pipeline.param[take_time]?' # show the value of a pipeline-wide parameter
-tweak 'analysis[add_together].analysis_capacity?' # show the value of an analysis attribute
-tweak 'analysis[add_together].param[foo]?' # show the value of an analysis parameter
-tweak 'resource_class[urgent].LSF?' # show the description of a particular meadow of a resource_class
Either pipeline-wide or analysis-wide parameters can also be deleted (the mnemonic for '#' is crossing-out) :
-tweak 'pipeline.param[foo]#' # delete a pipeline-wide parameter
-tweak 'analysis[add_together].param[bar]#' # delete an analysis-wide parameter
The new mechanism does not require that you prepare the PipeConfig files with $self->o() references, which significantly simplifies PipeConfigs.
New configuration mechanism to 'tweak' parameters and attributes of pipelines either during pipeline initialization or afterwards.
For tweaking things during initialization we have extended init_pipeline.pl to understand 'tweak' commands -SET , -SHOW and -DELETE.
For tweaking things after the pipeline database has been created there is a new tweak_pipeline.pl script that understands the same 'tweaks' :
-SET 'pipeline.param[take_time]=20' # override a value of a pipeline-wide parameter; can also create an inexistent parameter
-SET 'pipeline.hive_pipeline_name=new_name' # override a value of a hive_meta attribute
-SET 'analysis[take_b_apart].param[base]=10' # override a value of an analysis-wide parameter; can also create an inexistent parameter
-SET 'analysis[add_together].analysis_capacity=3' # override a value of an analysis attribute
-SET 'analysis[blast%].batch_size=15' # override a value of an analysis_stats attribute for all analyses matching a pattern
-SET 'analysis[part_multiply].resource_class=urgent' # set the resource class of an analysis (whether a resource class with this name existed or not)
-SET 'resource_class[urgent].LSF=-q yesteryear' # update or create a new resource description
In both contexts you can print out the current value of things:
-SHOW 'pipeline.hive_pipeline_name' # show the pipeline_name
-SHOW 'pipeline.param[take_time]' # show the value of a pipeline-wide parameter
-SHOW 'analysis[add_together].analysis_capacity' # show the value of an analysis attribute
-SHOW 'analysis[add_together].param[foo]' # show the value of an analysis parameter
-SHOW 'resource_class[urgent].LSF' # show the description of a particular meadow of a resource_class
Either pipeline-wide or analysis-wide parameters can also be deleted:
-DELETE 'pipeline.param[foo]' # delete a pipeline-wide parameter
-DELETE 'analysis[add_together].param[bar]' # delete an analysis-wide parameter
In addition to the simple attributes analyses also have two "complex" ones: wait_for and flow_into.
They can either be set from scratch:
-SET 'analysis[add_together].wait_for=["analysisX","analysisY"]' # remove all old wait_for rules, establish new ones
-SET 'analysis[part_multiply].flow_into={1=>"?table_name=intermediate_result"}' # remove all old flow_into rules, establish new ones
or you can append new ones to the existing pile of rules:
-SET 'analysis[add_together].wait_for+=["analysisZ","analysisW"]' # append two new wait_for rules
-SET 'analysis[part_multiply].flow_into+={1=>"another_sink"}' # append a new flow_into rule
You can only delete the whole set, not individually:
-DELETE 'analysis[add_together].wait_for' # delete all wait_for rules of an analysis
-DELETE 'analysis[part_multiply].flow_into' # delete all flow_into rules of an analysis
You can also check their current content:
-SHOW 'analysis[add_together].wait_for' # shows the list of wait_for rules of an analysis
-SHOW 'analysis[part_multiply].flow_into' # shows the list of flow_into rules of an analysis
The 'tweak' mechanism does not require that you prepare the PipeConfig files with $self->o() references, which significantly simplifies PipeConfigs.
Universal Runnables:
* JobFactory: non-contiguous split option has been added for those who have to use minibatching
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment