Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in
Toggle navigation
Open sidebar
ensembl-gh-mirror
ensembl-hive
Commits
aad3c9c8
Commit
aad3c9c8
authored
Apr 17, 2018
by
Matthieu Muffato
Browse files
Many corrections suggested by @ens-emily
parent
82fae692
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
21 additions
and
21 deletions
+21
-21
docs/creating_runnables/runnable_api.rst
docs/creating_runnables/runnable_api.rst
+21
-21
No files found.
docs/creating_runnables/runnable_api.rst
View file @
aad3c9c8
...
...
@@ -5,10 +5,10 @@ Runnable API
eHive exposes an interface for Runnables (jobs) to interact with the
system:
- query their own parameters
. S
ee :ref:`parameters-in-jobs`
- control its own execution and report issues
- run system commands
- trigger some *dataflow* events (e.g. create new jobs)
- query their own parameters
(s
ee :ref:`parameters-in-jobs`
),
- control its own execution and report issues
,
- run system commands
,
- trigger some *dataflow* events (e.g. create new jobs)
.
Reporting and logging
...
...
@@ -18,7 +18,7 @@ Jobs can log messages to the standard output with the
``$self->say_with_header($message, $important)`` method. However they are only printed
when the *debug* mode is enabled (see below) or when the ``$important`` flag is switched on.
They will also be prefixed with a standard prefix consisting of the
runtime context (
w
orker,
r
ole,
j
ob).
runtime context (
W
orker,
R
ole,
J
ob).
The debug mode is controlled by the ``--debug X`` option of
:ref:`script-beekeeper` and :ref:`script-runWorker`. *X* is an integer,
...
...
@@ -29,16 +29,16 @@ check whether it is 0 or not.
(so that the messages are printed on the standard output) but also stores
them in the database (in the ``log_message`` table).
To indicate that a
j
ob has to be terminated earlier (i.e. before reaching
To indicate that a
J
ob has to be terminated earlier (i.e. before reaching
the end of ``write_output``), you can call:
- ``$self->complete_early($message)`` to mark the
j
ob as *DONE*
- ``$self->complete_early($message)`` to mark the
J
ob as *DONE*
(successful run) and record the message in the database. Beware that this
will trigger the *autoflow*.
- ``$self->complete_early($message, $branch_code)`` is a variation of the
above that will replace the autoflow (branch 1) with a dataflow on the
branch given
- ``$self->throw($message)`` to log a failed attempt. The
j
ob may be given
branch given
.
- ``$self->throw($message)`` to log a failed attempt. The
J
ob may be given
additional retries following the analysis' *max_retry_count* parameter,
or is marked as *FAILED* in the database.
...
...
@@ -59,7 +59,7 @@ around this method).
meta-characters and delimiters such as ``>`` (to redirect the output to a
file), ``;`` (to separate two commands that have to be run sequentially)
or ``|`` (a pipe) and will be quoted and joined and passed to ``system``
as a single string
as a single string
.
#. An hashref of options. Accepted options are:
- ``use_bash_pipefail``: Normally, the exit status of a pipeline (e.g.
...
...
@@ -67,27 +67,27 @@ around this method).
errors in the first command are not captured. With the option turned
on, the exit status of the pipeline will capture errors in any command
of the pipeline, and will only be 0 if *all* the commands exit
successfully
successfully
.
- ``use_bash_errexit``: Exit immediately if a command fails. This is
mostly useful for cases like ``cmd1; cmd2`` where by default, ``cmd2``
would always be executed, regardless of the exit status of ``cmd1``
would always be executed, regardless of the exit status of ``cmd1``
.
- ``timeout``: the maximum number of seconds the command is allowed to
run for. The exit status will be set to -2 if the command had to be
aborted
aborted
.
During their execution, jobs may certainly have to use temporary files.
eHive provides a directory that will exist throughout the lifespan of the
w
orker with the ``$self->worker_temp_directory`` method. The directory is created
the first time the method is called, and deleted when the
w
orker ends. It is the Runnable's
W
orker with the ``$self->worker_temp_directory`` method. The directory is created
the first time the method is called, and deleted when the
W
orker ends. It is the Runnable's
responsibility to leave the directory in a clean-enough state for the next
j
ob (by removing some files, for instance), or to clean it up completely
J
ob (by removing some files, for instance), or to clean it up completely
with ``$self->cleanup_worker_temp_directory``.
By default, this directory will be put under /tmp, but it can be overriden
by adding a ``worker_temp_directory_name`` method to the runnable. This can
be used to:
- use a faster filesystem (although /tmp is usually local to the machine)
- use a faster filesystem (although /tmp is usually local to the machine)
,
- use a network filesystem (needed for distributed applications, e.g. over
MPI). See :ref:`worker_temp_directory_name-mpi` in the :ref:`howto-mpi` section.
...
...
@@ -107,9 +107,9 @@ $branch_number)`` method.
The payload ``$data`` must be of one of these types:
-
H
ash-reference that maps parameter names (strings) to their values
.
-
A
rray-reference of hash-references of the above type
- ``undef`` to propagate the job's input_id
-
h
ash-reference that maps parameter names (strings) to their values
,
-
a
rray-reference of hash-references of the above type
,
- ``undef`` to propagate the job's input_id
.
The branch number defaults to 1 and can be skipped. Generally speaking, it
has to be an integer.
...
...
@@ -121,6 +121,6 @@ to easily generate events. The method takes two arguments:
#. The path to a file containing one JSON object per line. Each line can be
prefixed with a branch number (and some whitespace), which will override
the default branch number.
#. The default branch number (defaults to 1 too)
#. The default branch number (defaults to 1 too)
.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment