Skip to content

pipeline_url quoting

Marek Szuba requested to merge bugfix/pipeline_url_quoting into version/2.5

Created by: muffato

Use case

@JAlvarezJarreta and @carlacummins found an issue in one of the Compara pipelines this morning. The pipeline defines a SystemCmd with the pipeline's URL, calling $self->pipeline_url() (see https://github.com/Ensembl/ensembl-compara/blob/release/97/modules/Bio/EnsEMBL/Compara/PipeConfig/Synteny_conf.pm#L139). When the pipeline is initialized by explictly giving -pipeline_url, the URL ends up being wrapped in quotes. It is thus not a valid URL any more.

$ init_pipeline.pl Bio::EnsEMBL::Compara::PipeConfig::EBI::Plants::Synteny_conf -pipeline_url 'mysql://ensadmin:xxxxx@mysql-ens-compara-prod-1.ebi.ac.uk:4485/muffato_plants_synteny_98'
$ tweak_pipeline.pl -url mysql://ensro@mysql-ens-compara-prod-1.ebi.ac.uk:4485/muffato_plants_synteny_98 -tweak 'analysis[1].param[cmd]?'
 
Tweak.Request   analysis[1].param[cmd]?
Tweak.Found     1 analyses matching the pattern '1'
Tweak.Show      analysis[populate_new_database].param[cmd] ::   ["#program#","--master","compara_master","--new","'mysql://ensadmin:xxxxx\@mysql-ens-compara-prod-1.ebi.ac.uk:4485/muffato_plants_synteny_98'","--reg-conf","#registry#"]

Description

The quotes are introduced by the hide_url_password function, see https://github.com/Ensembl/ensembl-hive/blob/version/2.5/modules/Bio/EnsEMBL/Hive/Utils/URL.pm#L308 and are removed by parse https://github.com/Ensembl/ensembl-hive/blob/version/2.5/modules/Bio/EnsEMBL/Hive/Utils/URL.pm#L45-L48 but the latter isn't called in all cases. In particular, $self->pipeline_url() in PipeConfigs returns the URL as given on the command-line, without parsing it (but after restoring the password https://github.com/Ensembl/ensembl-hive/blob/version/2.5/modules/Bio/EnsEMBL/Hive/Scripts/InitPipeline.pm#L42-L47). My change is to make the init_pipeline removes the quotes too.

Related to that, I've fixed a comment and the quoting of the URL when it is display in the "useful commands"

Possible Drawbacks

Maybe others didn't see this as a bug and are now expecting the URL to be wrapped in single-quotes ?

Testing

Have you added/modified unit tests to test the changes?

Yep

If so, do the tests pass/fail?

Yep

Have you run the entire test suite and no regression was detected?

Yes: ok.

Merge request reports