... | ... | @@ -4,10 +4,10 @@ All commands assume a bash environment. |
|
|
|
|
|
# The Quick Way (VM)
|
|
|
|
|
|
We provide a Vagrant (easy VirtualBox VM setups) machine spec currently available from https://github.com/andrewyatz/vagrant_machines/tree/master/ensembl/rest. To use this machine you should
|
|
|
We provide a Vagrant (easy VirtualBox VM setups) machine spec currently available from <https://github.com/andrewyatz/vagrant_machines/tree/master/ensembl/rest>. To use this machine you should
|
|
|
|
|
|
- Download and install VirtualBox from https://www.virtualbox.org/
|
|
|
- Download and install Vagrant from http://www.vagrantup.com/
|
|
|
- Download and install VirtualBox from <https://www.virtualbox.org/>
|
|
|
- Download and install Vagrant from <http://www.vagrantup.com/>
|
|
|
- Clone the vagrant machines repo and start it up like so
|
|
|
|
|
|
```bash
|
... | ... | @@ -26,7 +26,7 @@ You can change the version of REST installed by exporting ENSEMBL_VERSION before |
|
|
Ensure you have the following binaries available:
|
|
|
|
|
|
- Perl v5.14 upwards
|
|
|
- cvs
|
|
|
- Git
|
|
|
- bash
|
|
|
- wget/curl
|
|
|
|
... | ... | @@ -49,19 +49,19 @@ Your directory structure should look like this after installation: |
|
|
|
|
|
Information on installing Catalyst can be found at:
|
|
|
|
|
|
- http://wiki.catalystframework.org/wiki/installation (basic)
|
|
|
- http://wiki.catalystframework.org/wiki/installingcatalyst (advanced)
|
|
|
- <http://wiki.catalystframework.org/wiki/installation> (basic)
|
|
|
- <http://wiki.catalystframework.org/wiki/installingcatalyst> (advanced)
|
|
|
|
|
|
If you are on Debian/Ubuntu you can find all required packages using (remembering you will probably need a C compiler):
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
apt-get install build-essential
|
|
|
apt-cache search catalyst
|
|
|
```
|
|
|
|
|
|
If you are on Fedora/RedHat then the following should suffice:
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
yum groupinstall "Development Tools"
|
|
|
|
|
|
yum install perl-Catalyst-Runtime perl-Catalyst-Devel /usr/bin/catalyst.pl
|
... | ... | @@ -69,7 +69,7 @@ yum install perl-Catalyst-Runtime perl-Catalyst-Devel /usr/bin/catalyst.pl |
|
|
|
|
|
If you are using CPAN or CPANMINUS with local::lib then you can use:
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
cpanm Catalyst::Runtime Catalyst::Devel
|
|
|
```
|
|
|
|
... | ... | @@ -79,9 +79,9 @@ Please remember that the REST API has more dependencies than the ones mentioned |
|
|
|
|
|
A number of guides to installing the Ensembl API are available from:
|
|
|
|
|
|
- http://www.ensembl.org/info/docs/api/api_installation.html
|
|
|
- http://www.ensembl.org/info/docs/api/api_git.html
|
|
|
- http://www.ensembl.org/info/docs/webcode/install/ensembl-code.html
|
|
|
- <http://www.ensembl.org/info/docs/api/api_installation.html>
|
|
|
- <http://www.ensembl.org/info/docs/api/api_git.html>
|
|
|
- <http://www.ensembl.org/info/docs/webcode/install/ensembl-code.html>
|
|
|
|
|
|
Should you already have an Ensembl API for the required release then skip to "Installing the REST API".
|
|
|
|
... | ... | @@ -91,19 +91,25 @@ We will partially work through the second document to install the API. |
|
|
|
|
|
Use one of the following commands to get BioPerl
|
|
|
|
|
|
`wget http://bioperl.org/DIST/old_releases/bioperl-1.2.3.tar.gz`
|
|
|
```bash
|
|
|
wget http://bioperl.org/DIST/old_releases/bioperl-1.2.3.tar.gz
|
|
|
```
|
|
|
|
|
|
`curl -o bioperl-1.2.3.tar.gz http://bioperl.org/DIST/old_releases/bioperl-1.2.3.tar.gz`
|
|
|
```bash
|
|
|
curl -o bioperl-1.2.3.tar.gz http://bioperl.org/DIST/old_releases/bioperl-1.2.3.tar.gz
|
|
|
```
|
|
|
|
|
|
Now untar
|
|
|
|
|
|
`tar zxvf bioperl-1.2.3.tar.gz`
|
|
|
```bash
|
|
|
tar zxvf bioperl-1.2.3.tar.gz
|
|
|
```
|
|
|
|
|
|
### Installing Ensembl API from Tarball
|
|
|
|
|
|
We will download the Ensembl API from our stable branch tarball and unpack.
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
wget ftp://ftp.ensembl.org/pub/ensembl-api.tar.gz
|
|
|
tar zxvf ensembl-api.tar.gz
|
|
|
```
|
... | ... | @@ -114,7 +120,7 @@ tar zxvf ensembl-api.tar.gz |
|
|
|
|
|
This guide will checkout the stable current release (at the time of writing this is release/75 but this is updated every release)
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
git clone https://github.com/Ensembl/ensembl-rest.git
|
|
|
```
|
|
|
|
... | ... | @@ -122,7 +128,7 @@ git clone https://github.com/Ensembl/ensembl-rest.git |
|
|
|
|
|
Using bash we bring each modules directory onto the library path:
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
PERL5LIB=${PWD}/bioperl-1.2.3:${PERL5LIB}
|
|
|
PERL5LIB=${PWD}/ensembl/modules:${PERL5LIB}
|
|
|
PERL5LIB=${PWD}/ensembl-compara/modules:${PERL5LIB}
|
... | ... | @@ -135,21 +141,29 @@ Using bash we bring each modules directory onto the library path: |
|
|
|
|
|
First cd into the directory:
|
|
|
|
|
|
`cd ensembl-rest`
|
|
|
```bash
|
|
|
cd ensembl-rest
|
|
|
```
|
|
|
|
|
|
Then run cpanm asking it to install local dependencies
|
|
|
|
|
|
`cpanm --installdeps .`
|
|
|
```bash
|
|
|
cpanm --installdeps .
|
|
|
```
|
|
|
|
|
|
This will run for quite some time but will install all recommended & essential libraries. After which run Makefile.PL to confirm the setup:
|
|
|
|
|
|
`perl Makefile.PL`
|
|
|
```bash
|
|
|
perl Makefile.PL
|
|
|
```
|
|
|
|
|
|
### Installing with CPAN
|
|
|
|
|
|
Run the makefile:
|
|
|
|
|
|
`perl Makefile.PL`
|
|
|
```bash
|
|
|
perl Makefile.PL
|
|
|
```
|
|
|
|
|
|
This program will prompt to install CPAN dependencies required for the REST API. If you are on your machine as root and wish to do this into a central machine location then say Y to any prompt.
|
|
|
|
... | ... | @@ -161,11 +175,13 @@ Should you want to use another manager (such as apt or yum) you should use Makef |
|
|
|
|
|
First run Makefile.PL
|
|
|
|
|
|
`perl Makefile.PL`
|
|
|
```bash
|
|
|
perl Makefile.PL
|
|
|
```
|
|
|
|
|
|
This will emit output looking like:
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
|
|
|
|
|
include /Users/ayates/tmp/ensembl-rest/inc/Module/Install.pm
|
... | ... | @@ -217,13 +233,14 @@ Assuming your output shown no errors your installation is good. |
|
|
|
|
|
Secondly you should run `ping_ensembl.pl`
|
|
|
|
|
|
`perl ensembl/misc-scripts/ping_ensembl.pl`
|
|
|
```bash
|
|
|
perl ensembl/misc-scripts/ping_ensembl.pl
|
|
|
```
|
|
|
|
|
|
This script checks that all Ensembl dependencies are installed and we can contact the public Ensembl MySQL instances. If `ping_ensembl.pl` returns a successful message then your installation is ready to go.
|
|
|
|
|
|
# Configuration of the REST API
|
|
|
|
|
|
|
|
|
## Selecting the databases to use
|
|
|
|
|
|
The API offers two methods of configuring databases; using connections specified in the `ensembl_rest.conf` file or an Ensembl Registry file. We will use the former technique as it reduces the number of configuration files required.
|
... | ... | @@ -250,7 +267,7 @@ Firstly open `log4perl.conf` in a text editor & edit the first line accordingly |
|
|
|
|
|
Secondly open lib/EnsEMBL/REST.pm and edit the module to look like the following (search for use Catalyst to find the section we need to edit):
|
|
|
|
|
|
```
|
|
|
```perl
|
|
|
use Catalyst qw/
|
|
|
-Debug
|
|
|
ConfigLoader
|
... | ... | @@ -267,25 +284,27 @@ This will cause Catalyst to emit a lot of information about endpoints and the ro |
|
|
|
|
|
Catalyst allows you to specify a different configuration file to the default ensembl_rest.conf file held in the root of the Rest API. This is a useful feature when in production environments. Export the location of the config file as so:
|
|
|
|
|
|
`export ENSEMBL_REST_CONFIG=$APP_HOME/path/to/other/ensembl_rest.conf`
|
|
|
```bash
|
|
|
export ENSEMBL_REST_CONFIG=$APP_HOME/path/to/other/ensembl_rest.conf
|
|
|
```
|
|
|
|
|
|
See the following links for more information:
|
|
|
|
|
|
- https://metacpan.org/module/Catalyst::Plugin::ConfigLoader
|
|
|
- https://metacpan.org/module/Catalyst::Plugin::ConfigLoader::Manual
|
|
|
- <https://metacpan.org/module/Catalyst::Plugin::ConfigLoader>
|
|
|
- <https://metacpan.org/module/Catalyst::Plugin::ConfigLoader::Manual>
|
|
|
|
|
|
# Running The Server
|
|
|
|
|
|
## Using Catalyst's inbuilt server
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
cd ensembl-rest
|
|
|
./script/ensembl_rest_server.pl
|
|
|
```
|
|
|
|
|
|
This will start up a server with the output looking like:
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
2012/09/05 11:01:54 (0) Catalyst.pm 1184> EnsEMBL::REST powered by Catalyst 5.90015
|
|
|
HTTP::Server::PSGI: Accepting connections at http://0:3000/
|
|
|
```
|
... | ... | @@ -296,20 +315,20 @@ Navigate to the specified address and if everything has worked you should see th |
|
|
|
|
|
## Using a PSGI/Plack server
|
|
|
|
|
|
PSGI, http://plackperl.org/, is a framework independent way of programming HTTP based frameworks. We will run the REST server using starman, https://metacpan.org/module/Starman, and a PSGI file with 5 worker threads and running on port 3000 (same as the default Catalyst port).
|
|
|
PSGI, <http://plackperl.org/>, is a framework independent way of programming HTTP based frameworks. We will run the REST server using starman, <https://metacpan.org/module/Starman>, and a PSGI file with 5 worker threads and running on port 3000 (same as the default Catalyst port).
|
|
|
|
|
|
Starman can be installed using your system's default package management software or via cpan. Once installed you can invoke the server using the following:
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
cd ensembl-rest
|
|
|
PERL5LIB=$PWD/lib:$PERL5LIB starman --listen :3000 --workers 5 ensembl_rest.psgi
|
|
|
```
|
|
|
|
|
|
Navigate to either (depending on your machine):
|
|
|
|
|
|
- http://localhost:3000/
|
|
|
- http://127.0.0.1:3000/
|
|
|
- http://0.0.0.0:3000/
|
|
|
- <http://localhost:3000/>
|
|
|
- <http://127.0.0.1:3000/>
|
|
|
- <http://0.0.0.0:3000/>
|
|
|
|
|
|
Starman also has options for production environments including hot-deploy & daemonisation of the starman process. Please see its docs for more information.
|
|
|
|
... | ... | @@ -326,7 +345,7 @@ These are intended as a guide about how to run this server in a production mode |
|
|
|
|
|
Ensembl REST ships with a number of test databases. It is possible to run a local server which communicates only with these databases. Firstly you must setup the test case configuration like so
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
cd ensembl-rest
|
|
|
cp t/MultiTestDB.conf.example t/MultiTestDB.conf
|
|
|
$EDITOR t/MultiTestDB.conf
|
... | ... | @@ -336,7 +355,7 @@ $EDITOR t/MultiTestDB.conf |
|
|
|
|
|
To run execute the following
|
|
|
|
|
|
```
|
|
|
```bash
|
|
|
cd ensembl-rest
|
|
|
./scripts/ensembl_rest_test_server.pl
|
|
|
```
|
... | ... | @@ -346,12 +365,13 @@ The script supports the same debug, restart and file watching capacities as the |
|
|
|
|
|
# Errors & Tips
|
|
|
|
|
|
|
|
|
## Speeding up ID Lookup
|
|
|
|
|
|
A precomputed MySQL schema is available on USEast called ensembl_stable_ids_?? where ?? is the Ensembl release. This database is only available on USEast. Should you want a copy locally then please take a mysqldump of the schema or run the ensembl core script:
|
|
|
|
|
|
`ensembl/misc-scripts/stable_id_lookup/populate_stable_id_lookup.pl`
|
|
|
```bash
|
|
|
ensembl/misc-scripts/stable_id_lookup/populate_stable_id_lookup.pl
|
|
|
```
|
|
|
|
|
|
Documentation on the code is available from ensembl/misc-scripts/stable_id_lookup/README.
|
|
|
|
... | ... | @@ -373,21 +393,25 @@ This normally points to an error in your database setup. You should see a second |
|
|
|
|
|
Make sure your Host, Port and User attributes are set in your .conf file. If you are using a Registry file then ensure this is correctly configured and in the place it should be. If you want to use a relative path e.g. my/dir/reg.pm consider using the built-in Catalyst substitution macros.
|
|
|
|
|
|
`reg = my/dir/reg.pm`
|
|
|
```
|
|
|
reg = my/dir/reg.pm
|
|
|
```
|
|
|
|
|
|
Becomes:
|
|
|
|
|
|
`reg = _path_to(my/dir/reg.pm)__`
|
|
|
```
|
|
|
reg = _path_to(my/dir/reg.pm)__
|
|
|
```
|
|
|
|
|
|
More information is available from https://metacpan.org/module/Catalyst::Plugin::ConfigLoader
|
|
|
More information is available from <https://metacpan.org/module/Catalyst::Plugin::ConfigLoader>
|
|
|
|
|
|
## More information on Catalyst
|
|
|
|
|
|
You can see these websites for more Catalyst information:
|
|
|
|
|
|
- http://www.catalystframework.org/
|
|
|
- https://metacpan.org/module/Catalyst
|
|
|
- https://metacpan.org/module/Catalyst::Manual::Intro
|
|
|
- <http://www.catalystframework.org/>
|
|
|
- <https://metacpan.org/module/Catalyst>
|
|
|
- <https://metacpan.org/module/Catalyst::Manual::Intro>
|
|
|
|
|
|
## Caching vs. No Caching
|
|
|
|
... | ... | |