README.md 9.29 KB
Newer Older
Mihaly Varadi's avatar
Mihaly Varadi committed
1 2 3 4
# Welcome to the 3D-Beacons Registry

## Background

5 6 7 8 9
3D-Beacons is an open collaboration between providers of macromolecular structure models. The goal of this collaboration
is to provide model coordinates and meta-information from all the contributing data resources in a standardized data
format and on a unified platform.

![Image](https://www.ebi.ac.uk/pdbe/pdbe-kb/3dbeacons/assets/img/overview.png)
Mihaly Varadi's avatar
Mihaly Varadi committed
10

11
**Schematical overview of the 3D-Beacons infrastructure**
Mihaly Varadi's avatar
Mihaly Varadi committed
12

13 14 15 16 17
3D-Beacons consists of a Registry, a Hub and Beacons who host Clients. The Registry is used by
the [3D-Beacons Hub](https://github.com/3D-Beacons/3d-beacons-hub-api) to look up which API endpoints are supported by
the various Beacons. The Beacons provide data according to the 3D-Beacons data
specifications ([Current version: 0.3.1](https://app.swaggerhub.com/apis/3dbeacons/3D-Beacons/0.3.1)). The Hub collates
the data from the Beacons and expose it via Hub API endpoints.
Mihaly Varadi's avatar
Mihaly Varadi committed
18 19

### Current 3D-Beacons
20 21 22

- [AlphaFold](https://alphafold.ebi.ac.uk)
- [AlphaFill](https://alphafill.eu/)
Mihaly Varadi's avatar
Mihaly Varadi committed
23 24 25 26 27 28 29 30
- [Genome3D](http://genome3d.eu/)
- [Protein Data Bank in Europe](https://pdbe.org)
- [Protein Data Bank in Europe - Knowledge Base](https://pdbe-kb.org)
- [Protein Ensemble Database](https://proteinensemble.org/)
- [SWISS-MODEL](https://swissmodel.expasy.org/)

## About the 3D-Beacons Registry

31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
The 3D-Beacons Registry records meta-information about all the contributing partner resources, and lists the API
endpoints that they support. In other words, looking at the registry will give specific information on which API
endpoints provide what data from which data resource.

The Registry is implemented as a JSON object that complies with the schema specification, which is also included in this
repository.

These are available in `resources` folder in the repository. To add or change a registry entry, make the relevant
changes in `resources/registry.json` which should comply with schema defined in `resources/schema.json`.

There is also an installable Python package in this repository which provides utilities like schema validation. Please
follow the installation section below for installing the Python package.

## Linking a new data resource to 3D-Beacons Network

Data providers who are interested in making their macromolecule structures available through the 3D-Beacons Network
should follow the following steps:

1. Contact the 3D-Beacons consortium
2. Review the [API specifications](https://app.swaggerhub.com/apis/3dbeacons/3D-Beacons) for sharing metadata
3. Implement API endpoints or set up an instance of
   the [3D-Beacons Client](https://github.com/3D-Beacons/3d-beacons-client)
4. Review the `resources/registry.json` file in this repository
5. Update the `resources/registry.json` file to include information on your data resource and your API endpoint URLs
6. Create a pull request for the `development` branch with your updated `resources/registry.json` file

### 1. Contact the 3D-Beacons consortium

3D-Beacons is an open consortium, and we welcome new data providers who would like to make their experimentally
determined or theoretical macromolecule structures available through the 3D-Beacons Network.

To ensure that the network provides access to relevant data, we require new prospective data providers to contact us
before linking their data to 3D-Beacons. Please send an email to Sameer Velankar (sameer@ebi.ac.uk) or Christine
Orengo (c.orengo@ucl.ac.uk) to initiate discussions.

### 2. Review the [API specifications](https://app.swaggerhub.com/apis/3dbeacons/3D-Beacons) for sharing metadata

The 3D-Beacons Network provides access to metadata regarding macromolecule structures in a unified format. This means
that every data provider has to expose information in the same data format. We define the accepted data schemas in the
[3D-Beacons API specification](https://app.swaggerhub.com/apis/3dbeacons/3D-Beacons) on SwaggerHub.

Please review this specification, and identify the schemas that fit the data you would like to make accessible via
3D-Beacons. For example, if you want to make your structures discoverable based on a UniProt identifier, then the
endpoints with `/uniprot/{qualifier}.json` are relevant for you.

### 3. Implement API endpoints or set up an instance of the [3D-Beacons Client](https://github.com/3D-Beacons/3d-beacons-client)

After reviewing the API specifications and deciding what data you will make available, and which data schema you will
use, the next step is to either implement the selected API endpoints in a REST API, or to take advantage of
the [3D-Beacons Client](https://github.com/3D-Beacons/3d-beacons-client), which can be installed locally and includes a
81
pre-packaged and ready-to-use implementation of certain API endpoints. For more information on this, please visit the [3D-Beacons Client](https://github.com/3D-Beacons/3d-beacons-client) repository.
82 83 84

### 4. Review the `resources/registry.json` file in this repository

85
Once your metadata is exposed via API endpoints that comply with the [3D-Beacons API specification](https://app.swaggerhub.com/apis/3dbeacons/3D-Beacons), you should review the `resources/registry.json` file in this repository. This file contains all the information needed by the [3D-Beacons Hub API](https://github.com/3D-Beacons/3d-beacons-hub-api) for linking your API endpoints to the 3D-Beacons Network.
86 87 88

The registry has two main data blocks: 1.) `providers` and 2.) `services`.

89
The `providers` contains information that describes your data resource. We use this information to let users know where to look for the original sources of data.
Mihaly Varadi's avatar
Mihaly Varadi committed
90

91
An example item in the `providers` list would look like this:
92

93 94 95 96 97 98 99
```
{
   "providerId": "alphafold",
   "providerName": "AlphaFold Protein Structure Database",
   "providerDescription": "AlphaFold is an AI system developed by DeepMind that predicts a protein’s 3D structure from its amino acid sequence. It regularly achieves accuracy competitive with experiment.",
   "providerUrl": "https://alphafold.ebi.ac.uk/",
   "baseServiceUrl": "https://alphafold.ebi.ac.uk/api/",
100
   "devBaseServiceUrl": "https://dev.alphafold.ebi.ac.uk/api/",
101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116
   "providerLogo": "https://alphafold.ebi.ac.uk/assets/img/dm-logo.png"
}
```

The `services` contains information about what API endpoints are implemented by which data provider.

An example item in the `services` list would look like this:

```
{
   "serviceType": "summary",
   "provider": "alphafold",
   "accessPoint": "uniprot/summary/"
},
```

117
Together, the `providers` and `services` data blocks tell the 3D-Beacons Hub API that in the example above, AlphaFold DB provides access to their data by implementing the `summary` API endpoint, which they serve on the URL `https://alphafold.ebi.ac.uk/api/uniprot/summary/`
118 119 120 121 122

### 5. Update the `resources/registry.json` file

The next step is to fork this repository (
i.e. [https://github.com/3D-Beacons/3d-beacons-registry](https://github.com/3D-Beacons/3d-beacons-registry)) and edit
123 124 125
the `resources/registry.json` file by adding a new item in the `providers` list and listing all the API endpoints you implemented in the `services` list. 

**NOTE**: If you don't have a production setup at this point, set same value for `baseServiceUrl` as `devBaseServiceUrl`.
126 127 128

### 6. Create a pull request for the `development` branch

129
Finally, please create a pull request so that we can merge your version of the `resources/registry.json` file to our `development` branch. We will then test the updated file, and also test all the API endpoints you specified in the `services` list of the `resources/registry.json` file.
130

131
As part of testing the API endpoints, we will perform stress testing of all the API endpoints you provide. We will also validate the data format against the [3D-Beacons API specification](https://app.swaggerhub.com/apis/3dbeacons/3D-Beacons), and test if the [3D-Beacons Hub API](https://github.com/3D-Beacons/3d-beacons-hub-api) can concatenate data.
Mihaly Varadi's avatar
Mihaly Varadi committed
132

133
Once done, we proceed to merge the updates into the `master` branch, at which point your data resource will become officially linked to the 3D-Beacons Network.
134

135 136
## Installation

137 138
### Prerequisites

139
Below are the list of softwares/tools for the utilities to properly run in the environment.
140 141 142 143 144

Python 3.7+

**Note**

145 146 147
Because [Python 2.7 supports ended January 1](https://pythonclock.org/), 2020, new projects should consider supporting
Python 3 only, which is simpler than trying to support both. As a result, support for Python 2.7 in this project has
been dropped.
148 149

### Setup the environment
150

151
Setup a Python virtual environment and install required packages.
152

153 154 155 156 157 158 159 160 161 162 163 164
```
$ python3 -m venv venv
$ source venv/bin/activate
```

Now install the project dependencies.

```
(venv) $ make dev_deps
```

Install the package
165

166 167 168
```
(venv) $ pip install .
```
169 170 171

## Usage

172 173
The installed package can be used to validate `registry.json` against the defined schema. Both the files are available
in `resources` folder.
174 175 176 177

```
(venv) $ beacons_bio_3d validate_schema --schema_json resources/schema.json --registry_json resources/registry.json
```
178

Mihaly Varadi's avatar
Mihaly Varadi committed
179
## Contributors
180

181
- Sreenath Nair - _Initial work_ - [sreenathnair](https://github.com/sreenathnair)
Mihaly Varadi's avatar
Mihaly Varadi committed
182
- Mihaly Varadi - _Initial work_ - [mvaradi](https://github.com/mvaradi)
183

184 185
See also the list of [contributors](https://github.com/3D-Beacons/3d-beacons-registry/contributors) who participated in
this project.
186 187

### How to contribute
188

189
This repository is open to contributions. Please fork the repository and send pull requests.