index.rst

.. tsi-cc documentation master file, created by
   sphinx-quickstart on Mon Nov 12 16:26:08 2018.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.
 :orphan:

Welcome to Cloud Consultancy Team
=================================

This site is maintained by `the Cloud Consultancy Team at EBI <https://www.ebi.ac.uk/seqdb/confluence/display/TSI/Cloud+Consultancy+Team>`__

Table of contents
^^^^^^^^^^^^^^^^^

* `ResOps Training`_ - ResOps Training
* `Recommended Tools`_ - a list of recommended tools, and to get the most out of them
* `EBI Cloud Portal`_ - the EBI multi-cloud way of deploying applications
* `Architectural Design Patterns`_ - how to build cloud-native applications
* `About this documentation`_
* `Contact us`_
* `Site map`_

ResOps Training
^^^^^^^^^^^^^^^

* Notice

  * Please check the course pre-requisite on the agenda page.

* `Agenda for ResOps 2019 <ResOps/2019/Agenda-2019.html>`_
* `Agenda for ResOps 2020 <ResOps/2019/Agenda-2020-HD.html>`_

Recommended Tools
^^^^^^^^^^^^^^^^^

* Ansible:

..  * `Ansible best practice <bestpractices/Ansible/playbooks_best_practices.html>`_

* Cloud Consultancy Team toolbox:

  * `How to install tools <Tech-tips/Cloud-Consulting-Team-toolbox.html>`_

* Common Workflow Language

  * `CWL self-contained training course <https://www.melbournebioinformatics.org.au/tutorials/tutorials/cwl/cwl/>`_

* Docker:

  * `Docker best practice <Tech-tips/Tips-and-tricks-with-docker.html>`_

* GitLab:

  * Overview

    * `How to create docker image <Tech-tips/DevOps-toolchain-docker.html>`_
    * `How to deploy ECP instance <Tech-tips/DevOps-toolchain-ecp.html>`_
    * `How to publish documentation <Tech-tips/DevOps-toolchain-readthedocs.html>`_
    * `GitLab best practice <bestpractices/GitLab/README.html>`_

  * Useful links

    * `GitLab CI with Maven <https://www.ebi.ac.uk/seqdb/confluence/display/EXT/GitLab+CI+with+Maven>`_

.. * Grafana:

* Google Cloud Platform:

  * `Accessing GCP node from CLI <Tech-tips/Accessing-GCP-node-from-CLI.html>`_

* HPC in the cloud:

  * `HPC with Slurm on GCP <Tech-tips/HPC-with-Slurm-on-GCP.html>`_
  * Azure CycleCloud

    * `HPC with CycleCloud on Azure <Tech-tips/HPC-with-Azure-CycleCloud.html>`_
    * Useful links:

      * `CycleCloud Overview <https://docs.microsoft.com/en-us/azure/cyclecloud/>`_
      * `Setting up CycleCloud as a container <https://docs.microsoft.com/en-us/azure/cyclecloud/container-distribution>`_

* Kubernetes:

  * `Deployment of Kubernetes cluster on various clouds <Tech-tips/Deployment-of-Kubernetes-onto-clouds.html>`_
  * `Kubeflow for Machine Learning <Tech-tips/Kubeflow.html>`_
  * Useful links:

..  * `How to set up Container-as-a-service <Tech-tips/Installing-Docker-Kubernetes-for-CaaS.html>`_
..    * `CaaS Service Description <https://www.ebi.ac.uk/seqdb/confluence/display/CAAS/CaaS+Service+Description>`_

.. * Logstash (ELK):

* OneData:

  * `Overview <https://onedata.org/#/home/documentation/doc/getting_started/what_is_onedata.html>`_
  * `Oneclient <https://onedata.org/#/home/documentation/doc/using_onedata/oneclient.html>`_
  * `User Guide <https://onedata.org/#/home/documentation/doc/user_guide.html>`_
  * `Onedata API <https://onedata.org/#/home/api/latest/onezone>`_
  * EBI setup details
  * `EBI onedata portal <https://embl.tk>`_

.. * Prometheus:

* Terraform:

  * `Best Practice <Tech-tips/Tips-and-tricks-with-Terraform.html>`_

* Supervisor:

  * Description: Supervisor is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems.
  * `Reference documentation <http://supervisord.org/>`_

EBI Cloud Portal
^^^^^^^^^^^^^^^^

* `EMBL-EBI Cloud Portal documentation <https://ebi-cloud-portal.readthedocs.io/en/latest/>`_

  * `Using the EMBL-EBI Cloud Portal <https://ebi-cloud-portal.readthedocs.io/en/latest/using_the_portal.html>`_
  * `Packaging Applications for the EMBL-EBI Cloud Portal <https://ebi-cloud-portal.readthedocs.io/en/latest/applications_packaging.html>`_
  * `Avoid security credentials on git public repository <https://ebi-cloud-portal.readthedocs.io/en/latest/securing_credentials.html>`_
  * `API Endpoint documentation <https://ebi-cloud-portal.readthedocs.io/en/latest/api_endpoints.html>`_

Architectural Design Patterns
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* `How to convert monolithic design to microservices <architecture/monolithic_to_microservice/create_microservices.html>`_

.. Coming soon...
.. * Best practices
.. * Types of architecture approach at EBI
.. * Better way of doing
.. * Deployment design/architecture

About this documentation
^^^^^^^^^^^^^^^^^^^^^^^^
This documentation is `maintained at in a gitlab repository at EBI <https://gitlab.ebi.ac.uk/TSI/tsi-ccdoc>`_. Feel free to clone a local copy. You would need to run a local build to generate a local HTML site from the source files of `*.md` and `*.rst`. Install Sphinx with the instructions of `Getting started with sphinx <https://docs.readthedocs.io/en/latest/intro/getting-started-with-sphinx.html>`_. Follow the instructions of `Using markdown with sphinx <Tech-tips/DevOps-toolchain-readthedocs.html#using-markdown-with-sphinx>`_ and `Local build with Sphinx <Tech-tips/DevOps-toolchain-readthedocs.html#local-build-with-sphinx>`_ to generate your local site.

The **master** branch will always be the latest, most up-to-date version. Other versions may be available through the selection drop-down on the bottom-left corner of this page.

If you find any mistakes, have suggestions for improvement, have questions, or wish to talk to us about your project, please `contact us <mailto:cloud-consultants@ebi.ac.uk>`_. Pull requests or `issues filed against the git repository <https://gitlab.ebi.ac.uk/TSI/tsi-ccdoc/issues>`_ are also welcome.

Contact us
^^^^^^^^^^
You can `contact us by email <mailto:cloud-consultants@ebi.ac.uk>`_ or come find us in pod 14 of the South Building.

Site map
^^^^^^^^

.. toctree::
   :maxdepth: 2

   Presentation/Presentation.rst

   ResOps/2019/Agenda-2019.rst
   ResOps/2019/Demo-Creating-VMs-with-Terraform.rst
   ResOps/2019/Docker.md
   ResOps/2019/docker/01_running-containers.md
   ResOps/2019/docker/02_running-a-biobox-container.md
   ResOps/2019/docker/03_creating-a-docker-image.md
   ResOps/2019/docker/04_optimising-docker-images.md
   ResOps/2019/docker/05_using-the-builder-pattern.md
   ResOps/2019/docker/06_running-as-root.md
   ResOps/2019/docker/07_using-metadata.md
   ResOps/2019/docker/08_cleaning-up.md
   ResOps/2019/Gitlab.md
   ResOps/2019/gitlab/01_gitlab-account-setup.md
   ResOps/2019/gitlab/02_create-sample-project.md
   ResOps/2019/gitlab/03_run-docker-image.md
   ResOps/2019/gitlab/04_using-git-tags.md
   ResOps/2019/gitlab/05_add-further-steps.md
   ResOps/2019/gitlab/06_change-order-of-steps.md
   ResOps/2019/gitlab/07_pass-build-secrets.md
   ResOps/2019/gitlab/Error-tracking.rst
   ResOps/2019/gitlab/Github-pipeline.rst
   ResOps/2019/gitlab/Gitlab-DevOps.rst
   ResOps/2019/gitlab/Tracing.rst
   ResOps/2019/gitlab/auto-devops-tutorial.md
   ResOps/2019/Important-considerations-for-research-pipelines.rst
   ResOps/2019/Kubernetes-Demo-2019.rst
   ResOps/2019/Minikube-and-NGINX-Practical-2019.rst
   ResOps/2019/Scaling-up-Kubernetes.rst
   Tech-tips/Accessing-GCP-node-from-CLI.rst
   Tech-tips/Arvados-on-kubernetes.rst
   Tech-tips/Cloud-Consulting-Team-toolbox.md
   Tech-tips/DevOps-toolchain-docker.rst
   Tech-tips/DevOps-toolchain-ecp.rst
   Tech-tips/DevOps-toolchain-readthedocs.rst
   Tech-tips/HPC-with-Azure-CycleCloud.rst
   Tech-tips/HPC-with-Slurm-on-GCP.rst
   Tech-tips/Installing-openstack-cli-on-mac-os-x.md
   Tech-tips/Kubeflow.rst
   Tech-tips/Tips-and-tricks-with-docker.md
   Tech-tips/Tips-and-tricks-with-Terraform.md
.. Tech-tips/Deployment-of-Kubernetes-onto-clouds.rst
..   Tech-tips/Installing-Docker-Kubernetes-for-CaaS.rst
..   Rfam-cloud/Requirements-and-status-2018.11.08.md
..   Rfam-cloud/Requirements-and-status-2018.11.02.md
..   Rfam-cloud/Requirements-and-status-2018.10.26.md
..   Rfam-cloud/Requirements-and-status-2018.10.22.md
..   Rfam-cloud/Requirements-and-status-2018.10.09.md
..   Rfam-cloud/Prototype-of-nfs-and-k8s-cluster.md
..   Rfam-cloud/Quick-reference-for-rfam-cloud.md
..   Rfam-cloud/Rfam-tmp-results.md


.. Indices and tables
.. ==================

.. * :ref:`genindex`
.. * :ref:`modindex`
.. * :ref:`search`