when debugging an analysis which fails and would increment the retry_count.

to be promoted to 'DONE'

needed to better manage the hive system's load on the database housing all
the hive related tables (in case the database is overloaded by multiple users).
Added analysis_stats.sync_lock column (and correspondly in Object and Adaptor)
Added Queen::safe_synchronize_AnalysisStats method which wraps over the
  synchronize_AnalysisStats method and does various checks and locks to ensure
  that only one worker is trying to do a 'synchronize' on a given analysis at
  any given moment.
Cleaned up API between Queen/Worker so that worker only talks directly to the
  Queen, rather than getting the underlying database adaptor.
Added analysis_job columns runtime_msec, query_count to provide more data on
  how the jobs hammer a database (queries/sec).

called when worker dies to replace itself in the needed_workers count since
it's decremented when it's born, and it's counted as living (and subtracted)
as long as it's running.  This gunarantees that another worker will quickly
be created after this one dies (and not need to wait for a synch to happen)

is when there are lots of workers 'WORKING' so as to avoid them falling over
each other.  The 'WORKING' state only exists in the middle of a large run.
When the last worker dies, the state is 'ALL_CLAIMED' so the sync on death
will happen properly.  As the last pile of workers die they will all do
a synch, but that's OK since the system needs to be properly synched when
the last one dies since there won't be anybody left to do it.
Also added 10 minute check for if already 'SYNCHING' to deal with case if
worker dies in the middle of 'SYNCHING'.

so to reduce the sychronization frequency.

call lower down isn't needed.  Also needed to move the printing of the analysis_stats
up higher to better display with the new printing order.  Now -loop -analysis_stats looks right.

added check/set of status to 'SYNCHING' right before the synch procedure
so as to prevent multiple workers from simultaneously trying to synch at the same time.

will returns 0E0 if 'zero rows are inserted' which perl intreprets are true so I
need to check for it explicitly.  Also store method now returns 1 on 'new insert'
and '0' and 'already stored'.

complete an analysis.  If no job has been run (0 msec) it will assume 1 job
per worker up to the hive_capacity (maximum parallization).
Also changed worker->process_id to be the pid of the process not the ppid.

if it runs properly, the job looks like a normally claimed/fetched/run job

is asked to 're-run' a specific job.  By reclaiming, this job is properly
processed so when it finishes it looks like it was run normally by the system.

extended display in automation looping to print stats on currently running workers
and an overall statistic on the progress of the whole hive (% and total jobs)

  -debug <level> : turn on debug messages at <level>
  -no_cleanup    : don't perform global_cleanup when worker exits

better debug system to allow worker to control analysis debug level
new batch_size method will use avg_msec_per_job to calculate batch_size on the fly
  if analysis_stat.batch_size==0

keep analysis_job.input_id as varchar(255) to allow UNIQUE(analysis_id,input_id)
but in adaptor added logic so that if input_id in AnalysisJob object exceeds
the 255 char limit to store/fetch from the analysis_data table.  The input_id
in the analysis_job table becomes '_ext_input_analysis_data_id ##' which is a unique
internal variable to trigger the fetch routine to know to get the 'real' input_id
from the analysis_data table.
NO MORE 255 char limit on input_id and completely transparent to API user.

if the hash keys/values are the same.

to AnalysisStats object and store/fetch of avg_msec_per_job from analysis_stats table.
Corresponds to schema change.  Old hive databases need to do this sql to bring them up to spec;
  alter table analysis_stats add column avg_msec_per_job int(10) default 0 NOT NULL after batch_size;
  alter table analysis_stats modify column last_update datetime NOT NULL;

  analysis_stats.avg_msec_per_job int
modified
  analysis_stats.last_update to a datetime so that it is only changed when the API tells it to.

In run_autonomously method, updated a queen method call to print_running_worker_status. It was wrongly print_worker_status

to 'LOADING' to trigger sync so system knows that something changed

  -failed_jobs           : show all failed jobs
  -reset_job_id <num>    : reset a job back to READY so it can be rerun
  -reset_all_jobs_for_analysis_id <num>

added method reset_all_jobs_for_analysis_id to facilitate re-flowing data through new dataflow rules.
extended perldoc
changed the 'retry count' to 7 (so runs 1 + 7 retrys)