ensembl-io merge requestshttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests2019-06-18T08:53:37Zhttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/85Patch test databases [release/97]2019-06-18T08:53:37ZMarek SzubaPatch test databases [release/97]*Created by: mkszuba*
As the change set demonstrates, the last test-DB patching in this repository had taken place before all the necessary schema patches got published.*Created by: mkszuba*
As the change set demonstrates, the last test-DB patching in this repository had taken place before all the necessary schema patches got published.https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/86Release-98 version bump2019-06-18T08:54:16ZMarek SzubaRelease-98 version bump*Created by: mkszuba*
There is no API version number in _ensembl-io_ so all we have to do is patch the test databases. Includes the resync of the test ontology database with _ensembl_ as discussed in #85 .*Created by: mkszuba*
There is no API version number in _ensembl-io_ so all we have to do is patch the test databases. Includes the resync of the test ontology database with _ensembl_ as discussed in #85 .https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/76Check response status code instead of message for file fetch requests (for e94)2019-02-14T16:22:37ZMarek SzubaCheck response status code instead of message for file fetch requests (for e94)*Created by: imran-ebi*
This is the same as #74 for e94.*Created by: imran-ebi*
This is the same as #74 for e94.https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/77Check response status code instead of message for file fetch requests (for e96)2019-02-14T16:28:12ZMarek SzubaCheck response status code instead of message for file fetch requests (for e96)*Created by: imran-ebi*
This is the same as #74 for e96.*Created by: imran-ebi*
This is the same as #74 for e96.https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/83ancestral_allele update2019-04-23T10:11:10ZMarek Szubaancestral_allele update*Created by: tgrego*
Variation `Bio::EnsEMBL::Variation::Variation::ancestral_allele` method has been deprecated and replaced with `Bio::EnsEMBL::Variation::VariationFeature::ancestral_allele`.
`Bio::EnsEMBL::IO::Translator::SampleGe...*Created by: tgrego*
Variation `Bio::EnsEMBL::Variation::Variation::ancestral_allele` method has been deprecated and replaced with `Bio::EnsEMBL::Variation::VariationFeature::ancestral_allele`.
`Bio::EnsEMBL::IO::Translator::SampleGenotypeFeature` uses the deprecated method and is now failing, so an update is required.
This fixes the issue by patching the test DBs and the test data object, and updated the deprecated method to the new one.
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/81Backport of PR #802019-03-01T16:47:56ZMarek SzubaBackport of PR #80*Created by: tgrego*
This is a copy of
https://github.com/Ensembl/ensembl-io/pull/80
based against release/96
Will replace https://github.com/Ensembl/ensembl-io/pull/78
*Created by: tgrego*
This is a copy of
https://github.com/Ensembl/ensembl-io/pull/80
based against release/96
Will replace https://github.com/Ensembl/ensembl-io/pull/78
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/80Reinstate line deleted by accident2019-02-22T13:27:14ZMarek SzubaReinstate line deleted by accident*Created by: ens-ap5*
As per discussion with Mark & Tiago on 21/02/19*Created by: ens-ap5*
As per discussion with Mark & Tiago on 21/02/19https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/79ensembl Utils::IO into ensembl-io2019-03-04T16:00:48ZMarek Szubaensembl Utils::IO into ensembl-io*Created by: tgrego*
`Bio::EnsEMBL::Utils::IO::*` lives in the core repo at the moment.
This code predates the ensembl-io code, but it sure is IO related and should probably be located there.
This pull request moves that code into the...*Created by: tgrego*
`Bio::EnsEMBL::Utils::IO::*` lives in the core repo at the moment.
This code predates the ensembl-io code, but it sure is IO related and should probably be located there.
This pull request moves that code into the ensembl-io repo, keeping the same namespace. Thus code that uses it will not need any changes except that ensembl-io will now be a dependency.
This code was developed with the intention if being used internally by the production team, so it should be possible to let them know before branching of release/97.
Four filetypes were identified as being dealt with by both ensembl-io and utils-io: GTF, GFF, FASTA and BED.
Extra tests were added to check that the output of files produced by utils/io and ensembl-io are similar where they should be (t/utils_io/harmony.t). This is completed for the GTF and GFF formats (although only gene objects are tested at the moment... it's also easy to extend the test cases).
BED format seems however to be written in different formats by the 2 systems, thus there is only a stud for the test and more investigation required.
FASTA seems to be feature incomplete in ensembl-io (parser only), thus there is only a stud for the test and more investigation required (implementation of a FASTA writer needed?).
Utils/IO in the core repo can then possibly be deleted if this is working fine. I think the 4 release deprecation notice does not apply here as this is not really a deprecation but a move...
Namespace is the same, code is the same, all should be fine with the dependencies updated.
Coordinated with this https://github.com/Ensembl/ensembl/pull/368 has been submitted.
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/75Check response status code instead of message for file fetch requests (for e95)2019-02-15T11:00:30ZMarek SzubaCheck response status code instead of message for file fetch requests (for e95)*Created by: imran-ebi*
This is the same as #74 for e95.*Created by: imran-ebi*
This is the same as #74 for e95.https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/74Check response status code instead of message for file fetch requests2019-02-14T15:21:39ZMarek SzubaCheck response status code instead of message for file fetch requests*Created by: imran-ebi*
This fixes responses that have a message other than 'OK' (e.g. Gatewaying) failing. An example issue is, the Allele Frequency Calculator stopped submitting jobs to the hive as the FTP files fetched for it always ...*Created by: imran-ebi*
This fixes responses that have a message other than 'OK' (e.g. Gatewaying) failing. An example issue is, the Allele Frequency Calculator stopped submitting jobs to the hive as the FTP files fetched for it always return 'Gatewaying' as the reponse message. However, the status code was always 200 and the file was successfully fetched too.https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/63Fix of hard coded test data file paths in tests2018-08-13T10:14:35ZMarek SzubaFix of hard coded test data file paths in tests*Created by: tgrego*
## Description
Tests used an absolute path for the test files, so if test where run from a different location other than the root of the repo it would fail to find the files.
This changes the path to a relativ...*Created by: tgrego*
## Description
Tests used an absolute path for the test files, so if test where run from a different location other than the root of the repo it would fail to find the files.
This changes the path to a relative path, so that it always finds the test files.
## Use case
Tests are sometimes run from locations other than the root of the repo, and it would fail to find the test files.
## Benefits
Path for test files is now always correct.
## Possible Drawbacks
none I can think of.
## Testing
_Have you added/modified unit tests to test the changes?_
yes
_If so, do the tests pass/fail?_
pass
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/67Allow .bgz/.bgz.tbi files to be read in TabixParser2018-09-19T16:12:19ZMarek SzubaAllow .bgz/.bgz.tbi files to be read in TabixParser*Created by: ima23*
## Description
Some resources use _.bgz_ as ending for a BGZF-compressed vcf files (example gnomAD). Currently TabixParser allows only _.gz_ files to be read.
## Use case
The current changes will allow user...*Created by: ima23*
## Description
Some resources use _.bgz_ as ending for a BGZF-compressed vcf files (example gnomAD). Currently TabixParser allows only _.gz_ files to be read.
## Use case
The current changes will allow users to use _.vcf.bgz_ files for VEP custom annotations.
## Benefits
TabixParser will be able to process .bgz files and allow extended VEP custom annotations.
## Possible Drawbacks
_If applicable, describe any possible undesirable consequence of the changes._
## Testing
_Have you added/modified unit tests to test the changes?_
Yes.
_If so, do the tests pass/fail?_
Tests pass.
_Have you run the entire test suite and no regression was detected?_
Yes, tests passed.
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/68Merging of unified-design with master branches2018-12-17T16:17:22ZMarek SzubaMerging of unified-design with master branches*Created by: tgrego*
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the [Ens...*Created by: tgrego*
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the [Ensembl contributing guidelines](https://github.com/Ensembl/ensembl/blob/release/90/CONTRIBUTING.md#why-could-my-pull-request-be-rejected); remember in particular:
- do not modify code without testing for regression
- provide simple unit tests to test the changes
- the PR must not fail unit testing
## Description
This brings the feature/unified-design branch into master.
No conflicts exist, and all existing tests are passing (which is the essential functionality for web usage).
RDF support (both reading and writing) is included, so production does not have to checkout feature/unified-design branch to use it.
New writer design is in place, and support for writing GTF, GFF3 and VCF4 (in addition to RDF as mentioned before).
Writing of other filetypes (such as FASTA and BED) is not possible as that is not yet implemented.
Implementation and tests for missing writers is required!
Current master branch does not have writer tests, and ad hoc testing has shown it to be non-functional (thus no functionality is being lost with the merge).
## Benefits
RDF support and new writer design.
Unified master branch for a new round of development and feature implementation!
## Possible Drawbacks
Old writer design disappears so code that was writing files will not work anymore.
However, I could not find code that was writing files using that old design, and it does not seem to be working anyways...
## Testing
_Have you added/modified unit tests to test the changes?_
yes
_If so, do the tests pass/fail?_
pass
_Have you run the entire test suite and no regression was detected?_
yes
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/71porting of PR 70 to master2018-12-17T16:17:16ZMarek Szubaporting of PR 70 to master*Created by: tgrego*
see https://github.com/Ensembl/ensembl-io/pull/70*Created by: tgrego*
see https://github.com/Ensembl/ensembl-io/pull/70https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/72porting of PR 70 to release/952018-12-14T10:37:25ZMarek Szubaporting of PR 70 to release/95*Created by: tgrego*
see https://github.com/Ensembl/ensembl-io/pull/70*Created by: tgrego*
see https://github.com/Ensembl/ensembl-io/pull/70https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/61Improvements to error reporting - ENSWEB-20522018-08-01T11:58:15ZMarek SzubaImprovements to error reporting - ENSWEB-2052*Created by: ens-ap5*
## Description
Minor improvements to error reporting when attaching remote files
## Use case
Mainly for use when attaching trackhubs and parsing their files. See JIRA ENSWEB-2052.
## Benefits
Improve...*Created by: ens-ap5*
## Description
Minor improvements to error reporting when attaching remote files
## Use case
Mainly for use when attaching trackhubs and parsing their files. See JIRA ENSWEB-2052.
## Benefits
Improved usability
## Possible Drawbacks
None.
## Testing
Tested in browser. No applicable unit tests.https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/64[ENSCORESW-2690]. Close only if we've really got an iterator, can get…2018-09-06T10:50:07ZMarek Szuba[ENSCORESW-2690]. Close only if we've really got an iterator, can get…*Created by: sanjay-boddu*
… an empty hash ref under special cases.
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed ...*Created by: sanjay-boddu*
… an empty hash ref under special cases.
## Requirements
- Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
- Review the [Ensembl contributing guidelines](https://github.com/Ensembl/ensembl/blob/release/90/CONTRIBUTING.md#why-could-my-pull-request-be-rejected); remember in particular:
- do not modify code without testing for regression
- provide simple unit tests to test the changes
- the PR must not fail unit testing
## Description
This is a request to cherry pick one of the commits from release/93 to release/92.
## Use case
_Describe the problem. Please provide an example representing the motivation behind the need for having these changes in place._
## Benefits
_If applicable, describe the advantages the changes will have._
## Possible Drawbacks
_If applicable, describe any possible undesirable consequence of the changes._
## Testing
_Have you added/modified unit tests to test the changes?_
_If so, do the tests pass/fail?_
_Have you run the entire test suite and no regression was detected?_
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/66Allow .bgz/.bgz.tbi files to be read in TabixParser2018-09-19T16:16:18ZMarek SzubaAllow .bgz/.bgz.tbi files to be read in TabixParser*Created by: ima23*
## Description
Some resources use _.bgz_ as ending for a BGZF-compressed vcf files (example gnomAD). Currently TabixParser allows only _.gz_ files to be read.
## Use case
The current changes will allow user...*Created by: ima23*
## Description
Some resources use _.bgz_ as ending for a BGZF-compressed vcf files (example gnomAD). Currently TabixParser allows only _.gz_ files to be read.
## Use case
The current changes will allow users to use _.vcf.bgz_ files for VEP custom annotations.
## Benefits
TabixParser will be able to process .bgz files and allow extended VEP custom annotations.
## Possible Drawbacks
_If applicable, describe any possible undesirable consequence of the changes._
## Testing
_Have you added/modified unit tests to test the changes?_
Yes.
_If so, do the tests pass/fail?_
Tests pass.
_Have you run the entire test suite and no regression was detected?_
Yes, tests passed.
https://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/57Genbank has removed GI identifier in the genbank files2018-05-08T10:05:58ZMarek SzubaGenbank has removed GI identifier in the genbank files*Created by: thibauthourlier*
Do not look for GI in VERSION in the first regex. Still looking for GI if line has VERSION
Bug fix: Call get_accession instead of direct hash access to avoid accession to not be initialised*Created by: thibauthourlier*
Do not look for GI in VERSION in the first regex. Still looking for GI if line has VERSION
Bug fix: Call get_accession instead of direct hash access to avoid accession to not be initialisedhttps://gitlab.ebi.ac.uk/ensembl-gh-mirror/ensembl-io/-/merge_requests/56Add VCF4 format for validation2017-07-19T09:20:48ZMarek SzubaAdd VCF4 format for validation*Created by: Zhicheng-Liu*
Previoiusly because of the lack of VCF4 format definition, the user upload of VCF
type data (stored in text) could not be validated. This change is intended to add
this missing information so that those VCF ...*Created by: Zhicheng-Liu*
Previoiusly because of the lack of VCF4 format definition, the user upload of VCF
type data (stored in text) could not be validated. This change is intended to add
this missing information so that those VCF records could be validated and parsed.
Also add space as a valid delimiter in addition to tab. I know this is violating
the VCF4.x specs. But I could not get tab delimited VCF record to work. I don't
know why.
Related to ENSEMBL-5050.