GitLab

replaced rc_id by resource_class_id throughout the schema and added the foreign keys on resource_class_id

Removed maximise_concurrency and added analysis_stats.priority to guide the scheduler; improved scheduler and LSF meadow

big renaming of hive->worker, analysis_job->job, analysis_job_file->job_file, analysis_job_id->job_id, worker.beekeeper->worker.meadow_type, etc

testing the idea of analyses that can be empty for blocking purposes (it does not block while it is empty)

needed to better manage the hive system's load on the database housing all
the hive related tables (in case the database is overloaded by multiple users).
Added analysis_stats.sync_lock column (and correspondly in Object and Adaptor)
Added Queen::safe_synchronize_AnalysisStats method which wraps over the
  synchronize_AnalysisStats method and does various checks and locks to ensure
  that only one worker is trying to do a 'synchronize' on a given analysis at
  any given moment.
Cleaned up API between Queen/Worker so that worker only talks directly to the
  Queen, rather than getting the underlying database adaptor.
Added analysis_job columns runtime_msec, query_count to provide more data on
  how the jobs hammer a database (queries/sec).

called when worker dies to replace itself in the needed_workers count since
it's decremented when it's born, and it's counted as living (and subtracted)
as long as it's running.  This gunarantees that another worker will quickly
be created after this one dies (and not need to wait for a synch to happen)

to AnalysisStats object and store/fetch of avg_msec_per_job from analysis_stats table.
Corresponds to schema change.  Old hive databases need to do this sql to bring them up to spec;
  alter table analysis_stats add column avg_msec_per_job int(10) default 0 NOT NULL after batch_size;
  alter table analysis_stats modify column last_update datetime NOT NULL;

in AnalysisStatsAdaptor::fetch_by_analysis_id

the most time since last update are at the top of the returned list

than one status

created new() methods where needed, replaced throw, rearrange as needed

which causes a more even distribution of workers types as they are created by the Queen.

system load (higher hive_capacity) are picked first

decrement_needed_workers method. These are used by the Queen to pick an
analysis for a newly created worker when one wasn't specified

added method fetch_by_needed_workers for queen to determine new worker
types