GitLab

Individual Worker's specialization is now routed through Scheduler's main subroutine, to enforce more checks

schema_change: detect and register RELOCATED events that used to mess up things on LSF 9.0 ("job rescheduled" in LSF parlance)

be more careful with fetch_overdue_workers (Use 5sec threshold to avoid checking recently active Workers. Do not use it at all when performing -all_dead.)

bugfix:  last_check_in should only be updated by register_worker_death if the Worker is burying itself

Has one known limitation (always INSERTs instead of INSERT_IGNORE and REPLACE) and may contain other bugs.

finally implemented LSF's version of count_running_workers() and a Valley aggregator for all visible meadows

(currently they repeat a cycle of "wait t=1 sec, refresh stats" n=5 times,
maybe both of these parameters could be tuned better in future)

Back to num_required_workers' meaning "how many extra workers we need to add to this analysis"; fixing a scheduling bug/oversensitivity to manual change of batch_size

fixed a bug in computing num_required_workers according to the new rules (thanks Matthieu for reporting)

changed the meaning of 'num_required_workers' to "total estimated number of workers needed for this analysis" ( 'num_running_workers' is now included in it )

change of default behaviour: hive_capacity is now off by default (=NULL); setting hive_capacity=0 or analysis_capacity=0 stops scheduling AND specialization to a particular analysis

removed the "compile_module_once" option as the only way to compile modules now is once after specialization

Introduced a new 'NO_ROLE' cause_of_death for failures during specialization (not so much of an error, really!)

bugfix: avoid specializing in an otherwise BLOCKED analysis that is temporarily in SYNCHING state (thanks to Kathryn for reporting)

(multi-meadow scheduler) restrict the set of analyses that a worker with a given meadow_type can specialize into

Added API and schema support for analysis_base.meadow_type / Analysis->meadow_type(), which will be NULL/undef by default

make Valley into Configurable and move SubmitWorkersMax into Valley's context, because it is more "global" than a Meadow

capture Worker's death message during the new 'SPECIALIZATION' status in job_message/msg (thanks, Thomas!)

bugfix: make sure we release claimed jobs from a manually-run worker whose Runnable fails at compilation (thanks, Miguel!)

bugfix: query in Q::fetch_all_dead_workers_with_jobs() has to reference worker table by its full name

merge reset_and_grab into one subroutine; pre-increment dependent semaphore if re-running a DONE job; use -force flag for force-running an individual job