Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in
Toggle navigation
Open sidebar
ensembl-gh-mirror
ensembl-hive
Commits
8811e268
Commit
8811e268
authored
May 01, 2012
by
Leo Gordon
Browse files
a Process does not need a reference to the Queen
parent
d622eaf3
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
20 additions
and
25 deletions
+20
-25
modules/Bio/EnsEMBL/Hive/Process.pm
modules/Bio/EnsEMBL/Hive/Process.pm
+11
-16
modules/Bio/EnsEMBL/Hive/Worker.pm
modules/Bio/EnsEMBL/Hive/Worker.pm
+9
-9
No files found.
modules/Bio/EnsEMBL/Hive/Process.pm
View file @
8811e268
...
...
@@ -13,7 +13,7 @@
AnalysisCtrl rules.
Instances of these Processes are created by the system as work is done.
The newly created Process will have preset $self->
queen
, $self->dbc,
The newly created Process will have preset $self->
db
, $self->dbc,
$self->input_id, $self->analysis and several other variables.
From this input and configuration data, each Process can then proceed to
do something. The flow of execution within a Process is:
...
...
@@ -32,10 +32,10 @@
branch_codes. This is accomplished within the Process via
$self->dataflow_output_id(...);
The design philosophy is that each Process does it
'
s work and creates output,
The design philosophy is that each Process does its work and creates output,
but it doesn't worry about where the input came from, or where it's output
goes. If the system has dataflow pipes connected, then the output jobs
have purpose, if not the output work is thrown away. The workflow graph
have purpose, if not
-
the output work is thrown away. The workflow graph
'controls' the behaviour of the system, not the processes. The processes just
need to do their job. The design of the workflow graph is based on the knowledge
of what each Process does so that the graph can be correctly constructed.
...
...
@@ -208,22 +208,15 @@ sub DESTROY {
######################################################
=head2
queen
=head2
worker
Title :
queen
Usage : my $
hiveDBA
= $self->
queen
;
Function: returns the
'Queen'
this Process
was created
by
Title :
worker
Usage : my $
worker
= $self->
worker
;
Function: returns the
Worker object
this Process
is run
by
Returns : Bio::EnsEMBL::Hive::Queen
=cut
sub
queen
{
my
$self
=
shift
;
$self
->
{'
_queen
'}
=
shift
if
(
@
_
);
return
$self
->
{'
_queen
'};
}
sub
worker
{
my
$self
=
shift
;
...
...
@@ -231,6 +224,7 @@ sub worker {
return
$self
->
{'
_worker
'};
}
=head2 db
Title : db
...
...
@@ -243,7 +237,8 @@ sub worker {
sub
db
{
my
$self
=
shift
;
return
$self
->
queen
&&
$self
->
queen
->
db
;
$self
->
{'
_db
'}
=
shift
if
(
@
_
);
return
$self
->
{'
_db
'};
}
...
...
@@ -259,7 +254,7 @@ sub db {
sub
dbc
{
my
$self
=
shift
;
return
$self
->
queen
&&
$self
->
queen
->
dbc
;
return
$self
->
db
&&
$self
->
db
->
dbc
;
}
...
...
modules/Bio/EnsEMBL/Hive/Worker.pm
View file @
8811e268
...
...
@@ -366,7 +366,7 @@ sub worker_output_dir {
}
elsif
(
my
$hive_output_dir
=
$self
->
hive_output_dir
)
{
my
$worker_id
=
$self
->
dbID
()
;
my
$worker_id
=
$self
->
dbID
;
my
$dir_revhash
=
dir_revhash
(
$worker_id
);
$worker_output_dir
=
join
('
/
',
$hive_output_dir
,
dir_revhash
(
$worker_id
),
'
worker_id_
'
.
$worker_id
);
...
...
@@ -425,7 +425,7 @@ sub worker_process_temp_directory {
unless
(
defined
(
$self
->
{'
_tmp_dir
'})
and
(
-
e
$self
->
{'
_tmp_dir
'}))
{
#create temp directory to hold fasta databases
my
$username
=
$ENV
{'
USER
'};
my
$worker_id
=
$self
->
dbID
()
;
my
$worker_id
=
$self
->
dbID
;
$self
->
{'
_tmp_dir
'}
=
"
/tmp/worker_
${username}
.
${worker_id}
/
";
mkdir
(
$self
->
{'
_tmp_dir
'},
0777
);
throw
("
unable to create a writable directory
"
.
$self
->
{'
_tmp_dir
'})
unless
(
-
w
$self
->
{'
_tmp_dir
'});
...
...
@@ -590,7 +590,7 @@ sub run_one_batch {
$self
->
enter_status
('
COMPILATION
',
$job
);
# ToDo: when Runnables are ready, switch to compiling once per batch (saves time)
my
$runnable_db
=
$self
->
analysis
->
process
or
die
"
Unknown compilation error
";
$self
->
queen
->
dbc
->
query_count
(
0
);
$self
->
db
->
dbc
->
query_count
(
0
);
$job_stopwatch
->
restart
();
$self
->
run_module_with_job
(
$runnable_db
,
$job
);
...
...
@@ -598,7 +598,7 @@ sub run_one_batch {
my
$msg_thrown
=
$@
;
$job
->
runtime_msec
(
$job_stopwatch
->
get_elapsed
);
$job
->
query_count
(
$self
->
queen
->
dbc
->
query_count
);
$job
->
query_count
(
$self
->
db
->
dbc
->
query_count
);
my
$job_id
=
$job
->
dbID
();
my
$job_completion_line
=
"
\n
job
$job_id
: complete
\n
";
...
...
@@ -607,7 +607,7 @@ sub run_one_batch {
my
$job_status_at_the_moment
=
$job
->
status
();
my
$action
=
$job
->
incomplete
?
'
died
'
:
'
exited
';
$job_completion_line
=
"
\n
job
$job_id
:
$action
in status '
$job_status_at_the_moment
' for the following reason:
$msg_thrown
\n
";
$self
->
db
()
->
get_JobMessageAdaptor
()
->
register_message
(
$job_id
,
$msg_thrown
,
$job
->
incomplete
);
$self
->
db
->
get_JobMessageAdaptor
()
->
register_message
(
$job_id
,
$msg_thrown
,
$job
->
incomplete
);
}
print
STDERR
$job_completion_line
if
(
$self
->
worker_output_dir
and
(
$self
->
debug
or
$job
->
incomplete
));
# one copy goes to the job's STDERR
...
...
@@ -653,10 +653,10 @@ sub run_one_batch {
sub
run_module_with_job
{
my
(
$self
,
$runnable_db
,
$job
)
=
@_
;
$runnable_db
->
input_job
(
$job
);
$runnable_db
->
queen
(
$self
->
queen
);
$runnable_db
->
worker
(
$self
);
$runnable_db
->
debug
(
$self
->
debug
);
$runnable_db
->
input_job
(
$job
);
$runnable_db
->
db
(
$self
->
db
);
$runnable_db
->
worker
(
$self
);
$runnable_db
->
debug
(
$self
->
debug
);
$job
->
param_init
(
$runnable_db
->
strict_hash_format
(),
$runnable_db
->
param_defaults
(),
$self
->
db
->
get_MetaContainer
->
get_param_hash
(),
$self
->
analysis
->
parameters
(),
$job
->
input_id
()
);
$job
->
incomplete
(
1
);
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment