ensembl-hive merge requestshttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests2019-08-14T17:23:01Zhttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/136Cascade merge of PRs #134 and #135 into master2019-08-14T17:23:01ZMarek SzubaCascade merge of PRs #134 and #135 into master*Created by: ens-bwalts*
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the ...*Created by: ens-bwalts*
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the [development guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#ehive-development-in-a-nutshell) for eHive; remember in particular:
- Do not modify code without testing for regression.
- Provide simple unit tests to test the changes.
- If you change the database schema, please follow the instructions [for schema changes in the developer guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#schema-changes).
- If you change the schema, meadow, or guest language interfaces, please follow the scheme [for internal versioning in the developer guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#internal-versioning).
- The PR must not fail unit testing.
## Use case
Cascade merge of #134 and #135 into master
Note - as this is a cascade merge, it should be merged into master, not rebased.
## Description
See #134 and #135
## Possible Drawbacks
## Testing
Test-merged locally and no merge conflicts arose.
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/138Test using the latest Perl on Travis2019-08-13T10:09:32ZMarek SzubaTest using the latest Perl on Travis*Created by: muffato*
## Use case
Perl 5.14 not being available on Travis @ xenial, Ensembl as a whole is considering a bump of the minimal version (@s-mm brought this up to the ops meeting). All the teams have to start testing on th...*Created by: muffato*
## Use case
Perl 5.14 not being available on Travis @ xenial, Ensembl as a whole is considering a bump of the minimal version (@s-mm brought this up to the ops meeting). All the teams have to start testing on the latest Perl, which is 5.30
## Description
Just bumped the version in .travis.yml
## Possible Drawbacks
None
## Testing
_Have you added/modified unit tests to test the changes?_
N/A
_Have you run the entire test suite and no regression was detected?_
Yeshttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/137Add a "build" phase to the GuestLanguage interface2019-08-12T23:56:21ZMarek SzubaAdd a "build" phase to the GuestLanguage interface*Created by: muffato*
## Use case
The GuestLanguage interface requests a compilation of the wrapper every time a worker specializes into an analysis of that language. maven still taking ~10 seconds even if the wrapper is compiled and...*Created by: muffato*
## Use case
The GuestLanguage interface requests a compilation of the wrapper every time a worker specializes into an analysis of that language. maven still taking ~10 seconds even if the wrapper is compiled and up-to-date means that the test-suite is considerably slowed down. This model also doesn't scale as potentially hundreds of workers would all be calling maven at the same time.
## Description
In this PR I introduce a new phase of the GuestLanguage interface, "build", and a new script, {{refresh_guest_languages.pl}} to build all the wrappers. I also rename the "compile" phase to "check_exists". maven is not needed at runtime any more.
This allows some drastic changes to the Docker image. The Dockerfile is now a multi-stage build, maven being pulled in just 1 phase to build the wrapper. Then the final image only needs to include the JRE, not the JDK+maven. In fact there are now several more stages in the Dockerfile. This simplifies the handling of temporary tools that are only needed for one operation but not in the final image. I have also made the Dockerfile pull the whole checkout as "build context" rather than fetching it from Github (will have to reconfigure this on the Docker Hub). It greatly simplifies testing things locally and reduces the amount of data being transferred. That plus a few other changes, the eHive Docker image is now slightly more than 700MB (about a CD-ROM ^^). It was 1.12GB before
## Possible Drawbacks
The pipeline won' be able to automatically pick up changes in the wrapper code. One would have to run {{refresh_guest_languages.pl}}. I think this is acceptable and quite typical (like having to rerun `make` in order to update the executable).
## Testing
_Have you added/modified unit tests to test the changes?_
Already covered by tests.
_Have you run the entire test suite and no regression was detected?_
Yeshttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/133bugfix: this needs to match the source code2019-08-08T18:20:44ZMarek Szubabugfix: this needs to match the source code*Created by: muffato*
## Use case
At the moment beekeeper will report that the Java wrapper has an incompatible version.
## Description
The implementation is already compatible with the 4.* interface, but I forgot to change the...*Created by: muffato*
## Use case
At the moment beekeeper will report that the Java wrapper has an incompatible version.
## Description
The implementation is already compatible with the 4.* interface, but I forgot to change the version number in pom.xml in #110 / #111
## Possible Drawbacks
None
## Testing
_Have you added/modified unit tests to test the changes?_
No
_Have you run the entire test suite and no regression was detected?_
Yeshttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/134This should have stayed commented out2019-08-07T08:42:07ZMarek SzubaThis should have stayed commented out*Created by: muffato*
## Use case
When running an eHive docker container it will print "ripped a child" when exiting. This was meant to be a debug statement, mistakenly uncommented during #130
## Possible Drawbacks
None
## ...*Created by: muffato*
## Use case
When running an eHive docker container it will print "ripped a child" when exiting. This was meant to be a debug statement, mistakenly uncommented during #130
## Possible Drawbacks
None
## Testing
_Have you added/modified unit tests to test the changes?_
N/A
_Have you run the entire test suite and no regression was detected?_
Yeshttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/135com.fasterxml.jackson.core 2.9.9.1 has a vulnerability2019-08-07T08:42:05ZMarek Szubacom.fasterxml.jackson.core 2.9.9.1 has a vulnerability*Created by: muffato*
## Use case
See:
- https://github.com/Ensembl/ensembl-hive/network/alert/wrappers/java/pom.xml/com.fasterxml.jackson.core:jackson-databind/open
- https://nvd.nist.gov/vuln/detail/CVE-2019-14379
- https://nvd....*Created by: muffato*
## Use case
See:
- https://github.com/Ensembl/ensembl-hive/network/alert/wrappers/java/pom.xml/com.fasterxml.jackson.core:jackson-databind/open
- https://nvd.nist.gov/vuln/detail/CVE-2019-14379
- https://nvd.nist.gov/vuln/detail/CVE-2019-14439
## Description
Follow-up of #131. There could be more vulnerabilities, so I'm switching to an open interval to pick up the latest 2.9.9.* version
## Possible Drawbacks
_If applicable, describe any possible undesirable consequence of the changes._
## Testing
_Have you added/modified unit tests to test the changes?_
Nothing to change.
_Have you run the entire test suite and no regression was detected?_
Yeshttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/130bugfix: "returncode" is never set, so get the exit status from `waitpid`2019-07-30T09:07:38ZMarek Szubabugfix: "returncode" is never set, so get the exit status from `waitpid`*Created by: muffato*
## Use case
(reported by @tweep on the eHive-users mailing-list)
When running an eHive command through Docker, the container returns the exit code 0 even if there are some errors.
```
$ docker run -it ensem...*Created by: muffato*
## Use case
(reported by @tweep on the eHive-users mailing-list)
When running an eHive command through Docker, the container returns the exit code 0 even if there are some errors.
```
$ docker run -it ensemblorg/ensembl-hive seed_pipeline.pl --url mysql://user@blst.abc
Use of uninitialized value in concatenation (.) or string at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/DBSQL/CoreDBConnection.pm line 324.
Could not connect to database as user user using [DBI:mysql:host=blst.abc;port=3306] as a locator:
DBI connect('host=blst.abc;port=3306','user',...) failed: Unknown MySQL server host 'blst.abc' (0) at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/DBSQL/CoreDBConnection.pm line 317.
Use of uninitialized value in concatenation (.) or string at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/DBSQL/CoreDBConnection.pm line 333.
DB(mysql://user@blst.abc:3306/) Could not connect to database as user user using [DBI:mysql:host=blst.abc;port=3306] as a locator:
DBI connect('host=blst.abc;port=3306','user',...) failed: Unknown MySQL server host 'blst.abc' (0) at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/DBSQL/CoreDBConnection.pm line 317.
at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/DBSQL/CoreDBConnection.pm line 333.
Bio::EnsEMBL::Hive::DBSQL::CoreDBConnection::connect(Bio::EnsEMBL::Hive::DBSQL::DBConnection=HASH(0x1a10780)) called at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/DBSQL/DBConnection.pm line 139
eval {...} called at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/DBSQL/DBConnection.pm line 141
Bio::EnsEMBL::Hive::DBSQL::DBConnection::connect(Bio::EnsEMBL::Hive::DBSQL::DBConnection=HASH(0x1a10780)) called at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/DBSQL/CoreDBConnection.pm line 736
Bio::EnsEMBL::Hive::DBSQL::CoreDBConnection::db_handle(Bio::EnsEMBL::Hive::DBSQL::DBConnection=HASH(0x1a10780)) called at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/DBSQL/CoreDBConnection.pm line 976
Bio::EnsEMBL::Hive::DBSQL::CoreDBConnection::__ANON__(Bio::EnsEMBL::Hive::DBSQL::DBConnection=HASH(0x1a10780), undef, undef, "hive_meta") called at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/DBSQL/BaseAdaptor.pm line 243
Bio::EnsEMBL::Hive::DBSQL::BaseAdaptor::_table_info_loader(Bio::EnsEMBL::Hive::DBSQL::MetaAdaptor=HASH(0xa5e830)) called at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/DBSQL/BaseAdaptor.pm line 187
Bio::EnsEMBL::Hive::DBSQL::BaseAdaptor::column_set(Bio::EnsEMBL::Hive::DBSQL::MetaAdaptor=HASH(0xa5e830)) called at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/DBSQL/BaseAdaptor.pm line 628
Bio::EnsEMBL::Hive::DBSQL::BaseAdaptor::AUTOLOAD(Bio::EnsEMBL::Hive::DBSQL::MetaAdaptor=HASH(0xa5e830), "hive_sql_schema_version") called at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/DBSQL/DBAdaptor.pm line 128
eval {...} called at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/DBSQL/DBAdaptor.pm line 128
Bio::EnsEMBL::Hive::DBSQL::DBAdaptor::new("Bio::EnsEMBL::Hive::DBSQL::DBAdaptor", "-reg_alias", undef, "-reg_type", undef, "-url", "mysql://user\@blst.abc", "-no_sql_schema_version_check", undef, ...) called at /repo/ensembl-hive/modules/Bio/EnsEMBL/Hive/HivePipeline.pm line 181
Bio::EnsEMBL::Hive::HivePipeline::new("Bio::EnsEMBL::Hive::HivePipeline", "-url", "mysql://user\@blst.abc", "-reg_conf", undef, "-reg_type", undef, "-reg_alias", undef, ...) called at /repo/ensembl-hive/scripts/seed_pipeline.pl line 82
main::main() called at /repo/ensembl-hive/scripts/seed_pipeline.pl line 152
$ echo $?
0
```
## Description
It seems that `main_cmd.returncode` is not set (still _None_). According to the Python docs, I may have to call `main_cmd.wait()` but it didn't help. I suppose it's because the process has already been ripped (see the `waitpid` call above).
So my fix is to capture the main return code in `wait_for_all_processes` as well (alongside the other children's return codes) and return the first failure (the main process has the priority)
## Possible Drawbacks
Maybe some applications are used to eHive containers returning 0 ...
## Testing
_Have you added/modified unit tests to test the changes?_
We don't have any tests for the Docker image, so I've tested it locally:
```
docker run -it -v $PWD/scripts/dev/:/repo/ensembl-hive/scripts/dev/ ensemblorg/ensembl-hive seed_pipeline.pl --url mysql://user@blst.abc; echo $?
```
_Have you run the entire test suite and no regression was detected?_
The rest of the source code is not impactedhttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/132cascade merge of Docker return code and com.fasterxml.jackson.core fixes from...2019-07-29T13:36:54ZMarek Szubacascade merge of Docker return code and com.fasterxml.jackson.core fixes from 2.5 into master*Created by: ens-bwalts*
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the ...*Created by: ens-bwalts*
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the [development guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#ehive-development-in-a-nutshell) for eHive; remember in particular:
- Do not modify code without testing for regression.
- Provide simple unit tests to test the changes.
- If you change the database schema, please follow the instructions [for schema changes in the developer guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#schema-changes).
- If you change the schema, meadow, or guest language interfaces, please follow the scheme [for internal versioning in the developer guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#internal-versioning).
- The PR must not fail unit testing.
## Use case
Cascade merge of prs #130 and #131 into master. Note, as this PR is a step in a cascade merge, it should be merged into master, not rebased.
## Description
see #130 and #131
## Possible Drawbacks
see individual PRs
## Testing
_Have you added/modified unit tests to test the changes?_
_If so, do the tests pass/fail?_
_Have you run the entire test suite and no regression was detected?_
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/131com.fasterxml.jackson.core 2.9.9 has a vulnerability2019-07-29T13:03:09ZMarek Szubacom.fasterxml.jackson.core 2.9.9 has a vulnerability*Created by: muffato*
## Use case
See https://github.com/Ensembl/ensembl-hive/network/alert/wrappers/java/pom.xml/com.fasterxml.jackson.core:jackson-databind/open and https://nvd.nist.gov/vuln/detail/CVE-2019-12814 for a description ...*Created by: muffato*
## Use case
See https://github.com/Ensembl/ensembl-hive/network/alert/wrappers/java/pom.xml/com.fasterxml.jackson.core:jackson-databind/open and https://nvd.nist.gov/vuln/detail/CVE-2019-12814 for a description of the vulnerability.
I'm not aware of anyone using Java and eHive (I'm only aware of plans to do so) but I thought. Let's patch this and make GitHub happy.
## Description
Just bumped to 2.9.9.1. I don't want to define to an open interval as this is version/2.5, which shouldn't get any significant changes, and upstream may break the interface in a 3.* version.
## Possible Drawbacks
2.9.9.1 seems compatible, so I can't see any drawbacks
## Testing
_Have you added/modified unit tests to test the changes?_
Nothing to change.
_Have you run the entire test suite and no regression was detected?_
Yes. OKhttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/129Merge pipeline url quoting into master2019-07-28T18:18:12ZMarek SzubaMerge pipeline url quoting into master*Created by: ens-bwalts*
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the ...*Created by: ens-bwalts*
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the [development guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#ehive-development-in-a-nutshell) for eHive; remember in particular:
- Do not modify code without testing for regression.
- Provide simple unit tests to test the changes.
- If you change the database schema, please follow the instructions [for schema changes in the developer guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#schema-changes).
- If you change the schema, meadow, or guest language interfaces, please follow the scheme [for internal versioning in the developer guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#internal-versioning).
- The PR must not fail unit testing.
## Use case
Merges #129 into master
## Description
## Possible Drawbacks
Will create a bubble in the commit history due to previous mergeup of #108 leaving separate commits in both branches.
## Testing
See #129
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/128pipeline_url quoting2019-07-12T09:50:25ZMarek Szubapipeline_url quoting*Created by: muffato*
## Use case
@JAlvarezJarreta and @carlacummins found an issue in one of the Compara pipelines this morning.
The pipeline defines a SystemCmd with the pipeline's URL, calling `$self->pipeline_url()` (see https:...*Created by: muffato*
## Use case
@JAlvarezJarreta and @carlacummins found an issue in one of the Compara pipelines this morning.
The pipeline defines a SystemCmd with the pipeline's URL, calling `$self->pipeline_url()` (see https://github.com/Ensembl/ensembl-compara/blob/release/97/modules/Bio/EnsEMBL/Compara/PipeConfig/Synteny_conf.pm#L139). When the pipeline is initialized by explictly giving `-pipeline_url`, the URL ends up being wrapped in quotes. It is thus not a valid URL any more.
```
$ init_pipeline.pl Bio::EnsEMBL::Compara::PipeConfig::EBI::Plants::Synteny_conf -pipeline_url 'mysql://ensadmin:xxxxx@mysql-ens-compara-prod-1.ebi.ac.uk:4485/muffato_plants_synteny_98'
$ tweak_pipeline.pl -url mysql://ensro@mysql-ens-compara-prod-1.ebi.ac.uk:4485/muffato_plants_synteny_98 -tweak 'analysis[1].param[cmd]?'
Tweak.Request analysis[1].param[cmd]?
Tweak.Found 1 analyses matching the pattern '1'
Tweak.Show analysis[populate_new_database].param[cmd] :: ["#program#","--master","compara_master","--new","'mysql://ensadmin:xxxxx\@mysql-ens-compara-prod-1.ebi.ac.uk:4485/muffato_plants_synteny_98'","--reg-conf","#registry#"]
```
## Description
The quotes are introduced by the `hide_url_password` function, see https://github.com/Ensembl/ensembl-hive/blob/version/2.5/modules/Bio/EnsEMBL/Hive/Utils/URL.pm#L308 and are removed by `parse` https://github.com/Ensembl/ensembl-hive/blob/version/2.5/modules/Bio/EnsEMBL/Hive/Utils/URL.pm#L45-L48 but the latter isn't called in all cases. In particular, `$self->pipeline_url()` in PipeConfigs returns the URL as given on the command-line, without parsing it (but after restoring the password https://github.com/Ensembl/ensembl-hive/blob/version/2.5/modules/Bio/EnsEMBL/Hive/Scripts/InitPipeline.pm#L42-L47). My change is to make the init_pipeline removes the quotes too.
Related to that, I've fixed a comment and the quoting of the URL when it is display in the "useful commands"
## Possible Drawbacks
Maybe others didn't see this as a bug and are now expecting the URL to be wrapped in single-quotes ?
## Testing
_Have you added/modified unit tests to test the changes?_
Yep
_If so, do the tests pass/fail?_
Yep
_Have you run the entire test suite and no regression was detected?_
Yes: ok.https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/126Merge standaloneJob for guest languages into master2019-07-09T13:47:39ZMarek SzubaMerge standaloneJob for guest languages into master*Created by: ens-bwalts*
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the ...*Created by: ens-bwalts*
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the [development guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#ehive-development-in-a-nutshell) for eHive; remember in particular:
- Do not modify code without testing for regression.
- Provide simple unit tests to test the changes.
- If you change the database schema, please follow the instructions [for schema changes in the developer guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#schema-changes).
- If you change the schema, meadow, or guest language interfaces, please follow the scheme [for internal versioning in the developer guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#internal-versioning).
- The PR must not fail unit testing.
## Use case
See #108
## Description
This brings #108 into master. Because there are no merge conflicts to resolve, and there is only one branch to merge, this can be done via a pull request
## Possible Drawbacks
See #108
## Testing
See #108 and automated tests for this merge
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/127readability fixes2019-07-09T13:13:51ZMarek Szubareadability fixes*Created by: ens-bwalts*
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the ...*Created by: ens-bwalts*
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the [development guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#ehive-development-in-a-nutshell) for eHive; remember in particular:
- Do not modify code without testing for regression.
- Provide simple unit tests to test the changes.
- If you change the database schema, please follow the instructions [for schema changes in the developer guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#schema-changes).
- If you change the schema, meadow, or guest language interfaces, please follow the scheme [for internal versioning in the developer guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#internal-versioning).
- The PR must not fail unit testing.
## Use case
@mkszuba noted that we have an opportunity to make some readability fixes with the changes brought in with #108. We should make these fixes in 2.5 as well, as long as we're making them - we get a slightly cleaner commit history this way, too.
## Description
Added whitespace where it will add clarity, in lines that were new or being changed anyway.
## Possible Drawbacks
Authorship of functional change is one commit off on a couple of lines
## Testing
_Have you added/modified unit tests to test the changes?_
N/A
_If so, do the tests pass/fail?_
N/A
_Have you run the entire test suite and no regression was detected?_
Yeshttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/108standaloneJob for guest-languages2019-07-08T11:45:02ZMarek SzubastandaloneJob for guest-languages*Created by: muffato*
## Use case
I found his bug when testing #104
```standaloneJob.pl org.ensembl.hive.longmult.DigitFactory -language java -a_multiplier 1234 -b_multiplier 4567 -debug 1```
doesn't work unless you have manually ...*Created by: muffato*
## Use case
I found his bug when testing #104
```standaloneJob.pl org.ensembl.hive.longmult.DigitFactory -language java -a_multiplier 1234 -b_multiplier 4567 -debug 1```
doesn't work unless you have manually compiled the Java code. The compilation only happens automatically when running init_pipeline.pl or runWorker.pl
## Description
I'm making standaloneJob even closer to runWorker by asking the Worker itself to compile the module (after creating a Role that links the Worker to the Analysis), like in the other parts of eHive.
## Possible Drawbacks
N/A
## Testing
_Have you added/modified unit tests to test the changes?_
I will add a test soon
_If so, do the tests pass/fail?_
_Have you run the entire test suite and no regression was detected?_
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/120Process/master merge 2.52019-07-04T18:04:34ZMarek SzubaProcess/master merge 2.5*Created by: ens-bwalts*
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the ...*Created by: ens-bwalts*
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the [development guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#ehive-development-in-a-nutshell) for eHive; remember in particular:
- Do not modify code without testing for regression.
- Provide simple unit tests to test the changes.
- If you change the database schema, please follow the instructions [for schema changes in the developer guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#schema-changes).
- If you change the schema, meadow, or guest language interfaces, please follow the scheme [for internal versioning in the developer guidelines](https://ensembl-hive.readthedocs.io/en/master/dev/development_guidelines.html#internal-versioning).
- The PR must not fail unit testing.
## Use case
This PR will complete the cascade merge into master of two recent PRs incorporated into version/2.5
## Description
This incorporates #111 #113 #115 #118
## Possible Drawbacks
This should preserve our commit history structure, but I've never tried this before.
## Testing
_Have you added/modified unit tests to test the changes?_
See relevant PRs
_If so, do the tests pass/fail?_
See relevant PRs
_Have you run the entire test suite and no regression was detected?_
Yes - this branch was created explicitly to force a Travis build to check that changes in .travis.yml worked when merged onto masterhttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/98Version/2.32019-07-04T12:18:24ZMarek SzubaVersion/2.3*Created by: ens-bwalts*
Cascade merge*Created by: ens-bwalts*
Cascade mergehttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/121Exclude "/" from pipeline names too as it causes problems in URLs2019-07-03T10:55:40ZMarek SzubaExclude "/" from pipeline names too as it causes problems in URLs*Created by: muffato*
## Use case
Jorge found this one by accident. He had put a `/` in the `pipeline_name` option and then generate_graph.pl failed with this error
```Error: <stdin>: syntax error in line 119 near '/'```
When d...*Created by: muffato*
## Use case
Jorge found this one by accident. He had put a `/` in the `pipeline_name` option and then generate_graph.pl failed with this error
```Error: <stdin>: syntax error in line 119 near '/'```
When doing my own tests, I realized that the URLs themselves would be confusing with a `/` in the database name, so better to remove them early on.
## Description
Just like whitespace, replace the `/` with an underscore
## Possible Drawbacks
N/A
## Testing
_Have you added/modified unit tests to test the changes?_
No, that function is not currently tested.
_If so, do the tests pass/fail?_
_Have you run the entire test suite and no regression was detected?_
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/116[codecov] Only report a failure when the coverage decreases by more than 1%2019-07-03T10:55:16ZMarek Szuba[codecov] Only report a failure when the coverage decreases by more than 1%*Created by: muffato*
## Use case
I've always found it very annoying that codecov complains about meaningless decreases of the coverage, like `80.88% (-0.04%) compared to e2e90e5 `. Coveralls has been configured to not report small...*Created by: muffato*
## Use case
I've always found it very annoying that codecov complains about meaningless decreases of the coverage, like `80.88% (-0.04%) compared to e2e90e5 `. Coveralls has been configured to not report small differences but I couldn't find how to configure this in codecov ... until today :)
## Description
Do not complain about small decreases of the coverage
## Possible Drawbacks
Does anyone need such precision and fine-grain decision in the codecov report ?
## Testing
_Have you added/modified unit tests to test the changes?_
N/A, but I have checked that the yaml file is valid with
```
curl --data-binary @.codecov.yml https://codecov.io/validate
```
_Have you run the entire test suite and no regression was detected?_
N/Ahttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/119Report the command that is failing2019-07-02T10:37:15ZMarek SzubaReport the command that is failing*Created by: muffato*
## Use case
Since 2.5, the commands being run are by default hidden (they can be shown with the `-hive_debug_init 1` flag), so if something goes wrong, you can only see the error message, not the command that ca...*Created by: muffato*
## Use case
Since 2.5, the commands being run are by default hidden (they can be shown with the `-hive_debug_init 1` flag), so if something goes wrong, you can only see the error message, not the command that caused the error.
## Description
When the command fails, print it too, alongside the error message.
## Possible Drawbacks
N/A
## Testing
_Have you added/modified unit tests to test the changes?_
N/A
_If so, do the tests pass/fail?_
_Have you run the entire test suite and no regression was detected?_
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-hive/-/merge_requests/115JDK version2019-07-01T15:53:43ZMarek SzubaJDK version*Created by: muffato*
## Use case
The 2.5 Docker image cannot run the Java wrapper since it doesn't have maven/openjdk installed.
## Description
Various commits, related to the Docker image and the JDK version:
- use OpenJDK 1...*Created by: muffato*
## Use case
The 2.5 Docker image cannot run the Java wrapper since it doesn't have maven/openjdk installed.
## Description
Various commits, related to the Docker image and the JDK version:
- use OpenJDK 12 as it is the only released version that has the necessary dependencies (on both Travis and Docker)
- minor cleanup of the Docker image
By the way, if you have written any update to the doc / install guide for OpenJDK, can you add it to this branch ? If not, I will add something now
## Possible Drawbacks
N/A
## Testing
_Have you added/modified unit tests to test the changes?_
Not needed
_Have you run the entire test suite and no regression was detected?_
Yes. OK