Commit 61421dcf authored by Leo Gordon's avatar Leo Gordon
Browse files

updated schema diagram and docs for schema v.60

parent 7964fa0d
......@@ -130,10 +130,11 @@ You can toggle the display of individual columns using [Show/Hide columns] butto
<div style="background-color:#F2F2F2;border-radius:5px;margin-bottom:15px;float:left;margin-right:20px;width:200px">
<div style="padding:2px 5px;background-color:#336;border-top-left-radius:5px;border-top-right-radius:5px">
<div style="background-color:#24DA06;border:1px solid #FFF;padding:0px 8px;display:inline;vertical-align:middle"></div>
<h2 style="margin-left:8px;display:inline;color:#FFF;vertical-align:middle">worker table</h2>
<h2 style="margin-left:8px;display:inline;color:#FFF;vertical-align:middle">execution tables</h2>
</div>
<ul style="padding:0px 4px 0px 22px;margin-bottom:2px">
<li><a href="#worker"><b>worker</b></a></li>
<li><a href="#role"><b>role</b></a></li>
</ul>
</div>
......@@ -479,7 +480,7 @@ You can toggle the display of individual columns using [Show/Hide columns] butto
<br /><br />
<div style="background-color:#F2F2F2;padding:5px;margin:5px 0px;border-top:2px solid #24DA06">
<h2 id="4" style="display:inline;color:#000">worker table</h2>
<h2 id="4" style="display:inline;color:#000">execution tables</h2>
</div>
<div id="worker" style="width:820px;height:20px;border: 2px groove #CCC;background-color:#FAFAFF;padding:2px;margin-top:35px;margin-bottom:2px">
......@@ -515,6 +516,32 @@ You can toggle the display of individual columns using [Show/Hide columns] butto
</table>
</div>
<div id="role" style="width:820px;height:20px;border: 2px groove #CCC;background-color:#FAFAFF;padding:2px;margin-top:35px;margin-bottom:2px">
<div style="float:left;padding:0px;height:20px;width:10px;background-color:#24DA06;margin-right:10px"></div>
<div style="float:left;text-align:left;font-size:11pt;font-weight:bold">role</div>
<div style="float:right;text-align:right;padding-right:1px">
<a id="a_role" style="cursor:pointer;text-decoration:none" onclick="show_hide('role')">
<img src="/i/16/plus-button.png" style="width:12px;height:12px;vertical-align:middle" alt="show"/>
<span style="vertical-align:middle">Show columns</span>
</a>
<b> | </b> <a href="#top">[Back to top]</a>
</div>
</div>
<p style="padding:5px 0px;margin-bottom:0px;width:800px">Entries of this table correspond to Role objects of the API. When a Worker specializes, it acquires a Role, which is a temporary link between the Worker and a resource-compatible Analysis.</p>
<div id="div_role" style="display:none">
<table style="border:1px solid #667aa6;padding:0px;min-width:1000px;max-width:1200px">
<tr class="center" style="color:#FFFFFF;background-color:#667aa6"><th style="color:#FFF;padding:2px">Column</th><th style="color:#FFF;padding:2px">Type</th><th style="color:#FFF;padding:2px;min-width:80px">Default value</th><th style="color:#FFF;padding:2px;min-width:500px">Description</th><th style="color:#FFF;padding:2px;min-width:100px">Index</th></tr>
<tr class="bg1"><td><b>role_id</b></td><td>INTEGER</td><td></td><td>unique ID of the Role</td><td></td></tr>
<tr class="bg2"><td><b>worker_id</b></td><td>INTEGER</td><td></td><td>the specialized Worker</td><td>key: worker</td></tr>
<tr class="bg1"><td><b>analysis_id</b></td><td>INTEGER</td><td></td><td>the Analysis into which the Worker specialized</td><td>key: analysis</td></tr>
<tr class="bg2"><td><b>when_started</b></td><td>TIMESTAMP</td><td>CURRENT_TIMESTAMP</td><td>when this Role started</td><td></td></tr>
<tr class="bg1"><td><b>when_finished</b></td><td>TIMESTAMP</td><td></td><td>when this Role finished. NULL may either indicate it is still running or was killed by an external force.</td><td></td></tr>
<tr class="bg2"><td><b>attempted_jobs</b></td><td>INTEGER</td><td>0</td><td>counter of the number of attempts</td><td></td></tr>
<tr class="bg1"><td><b>done_jobs</b></td><td>INTEGER</td><td>0</td><td>counter of the number of successful attempts</td><td></td></tr>
</table>
</div>
<br /><br />
<div style="background-color:#F2F2F2;padding:5px;margin:5px 0px;border-top:2px solid #F4D20C">
<h2 id="5" style="display:inline;color:#000">Logging and monitoring</h2>
......@@ -537,13 +564,13 @@ You can toggle the display of individual columns using [Show/Hide columns] butto
<table style="border:1px solid #667aa6;padding:0px;min-width:1000px;max-width:1200px">
<tr class="center" style="color:#FFFFFF;background-color:#667aa6"><th style="color:#FFF;padding:2px">Column</th><th style="color:#FFF;padding:2px">Type</th><th style="color:#FFF;padding:2px;min-width:80px">Default value</th><th style="color:#FFF;padding:2px;min-width:500px">Description</th><th style="color:#FFF;padding:2px;min-width:100px">Index</th></tr>
<tr class="bg1"><td><b>worker_id</b></td><td>INTEGER</td><td></td><td>links to the worker table</td><td>primary key</td></tr>
<tr class="bg2"><td><b>exit_status</b></td><td>VARCHAR(255)</td><td></td><td>meadow-dependent, in case of LSF it's usually 'done' (normal) or 'exit' (abnormal)</td><td></td></tr>
<tr class="bg1"><td><b>mem_megs</b></td><td>FLOAT</td><td></td><td>how much memory the Worker process used</td><td></td></tr>
<tr class="bg2"><td><b>swap_megs</b></td><td>FLOAT</td><td></td><td>how much swap the Worker process used</td><td></td></tr>
<tr class="bg1"><td><b>pending_sec</b></td><td>INTEGER</td><td></td><td>time spent by the process in the queue before it became a Worker</td><td></td></tr>
<tr class="bg2"><td><b>cpu_sec</b></td><td>FLOAT</td><td></td><td>cpu time used by the Worker process</td><td></td></tr>
<tr class="bg1"><td><b>lifespan_sec</b></td><td>INTEGER</td><td></td><td>walltime used by the Worker process</td><td></td></tr>
<tr class="bg2"><td><b>exception_status</b></td><td>VARCHAR(255)</td><td></td><td>meadow-specific flags, in case of LSF it can be 'underrun', 'overrun' or 'idle'</td><td></td></tr>
<tr class="bg2"><td><b>exit_status</b></td><td>VARCHAR(255)</td><td>NULL</td><td>meadow-dependent, in case of LSF it's usually 'done' (normal) or 'exit' (abnormal)</td><td></td></tr>
<tr class="bg1"><td><b>mem_megs</b></td><td>FLOAT</td><td>NULL</td><td>how much memory the Worker process used</td><td></td></tr>
<tr class="bg2"><td><b>swap_megs</b></td><td>FLOAT</td><td>NULL</td><td>how much swap the Worker process used</td><td></td></tr>
<tr class="bg1"><td><b>pending_sec</b></td><td>FLOAT</td><td>NULL</td><td>time spent by the process in the queue before it became a Worker</td><td></td></tr>
<tr class="bg2"><td><b>cpu_sec</b></td><td>FLOAT</td><td>NULL</td><td>cpu time used by the Worker process</td><td></td></tr>
<tr class="bg1"><td><b>lifespan_sec</b></td><td>FLOAT</td><td>NULL</td><td>walltime used by the Worker process</td><td></td></tr>
<tr class="bg2"><td><b>exception_status</b></td><td>VARCHAR(255)</td><td>NULL</td><td>meadow-specific flags, in case of LSF it can be 'underrun', 'overrun' or 'idle'</td><td></td></tr>
</table>
</div>
......@@ -565,12 +592,13 @@ You can toggle the display of individual columns using [Show/Hide columns] butto
<tr class="center" style="color:#FFFFFF;background-color:#667aa6"><th style="color:#FFF;padding:2px">Column</th><th style="color:#FFF;padding:2px">Type</th><th style="color:#FFF;padding:2px;min-width:80px">Default value</th><th style="color:#FFF;padding:2px;min-width:500px">Description</th><th style="color:#FFF;padding:2px;min-width:100px">Index</th></tr>
<tr class="bg1"><td><b>log_message_id</b></td><td>INTEGER</td><td></td><td>an autoincremented primary id of the message</td><td></td></tr>
<tr class="bg2"><td><b>job_id</b></td><td>INTEGER</td><td>NULL</td><td>the id of the job that threw the message (or NULL if it was outside of a message)</td><td>key: job_id</td></tr>
<tr class="bg1"><td><b>worker_id</b></td><td>INTEGER</td><td>NULL</td><td>the 'current' worker</td><td>key: worker_id</td></tr>
<tr class="bg2"><td><b>time</b></td><td>TIMESTAMP</td><td>CURRENT_TIMESTAMP</td><td>when the message was thrown</td><td></td></tr>
<tr class="bg1"><td><b>retry</b></td><td>INTEGER</td><td>NULL</td><td>retry_count of the job when the message was thrown (or NULL if no job)</td><td></td></tr>
<tr class="bg2"><td><b>status</b></td><td>ENUM('UNKNOWN','SPECIALIZATION','COMPILATION','CLAIMED','READY','PRE_CLEANUP','FETCH_INPUT','RUN','WRITE_OUTPUT','POST_CLEANUP','PASSED_ON')</td><td>'UNKNOWN'</td><td>of the job or worker when the message was thrown</td><td></td></tr>
<tr class="bg1"><td><b>msg</b></td><td>TEXT</td><td></td><td>string that contains the message</td><td></td></tr>
<tr class="bg2"><td><b>is_error</b></td><td>SMALLINT</td><td></td><td>binary flag</td><td></td></tr>
<tr class="bg1"><td><b>role_id</b></td><td>INTEGER</td><td>NULL</td><td>the 'current' role</td><td></td></tr>
<tr class="bg2"><td><b>worker_id</b></td><td>INTEGER</td><td>NULL</td><td>the 'current' worker</td><td>key: worker_id</td></tr>
<tr class="bg1"><td><b>time</b></td><td>TIMESTAMP</td><td>CURRENT_TIMESTAMP</td><td>when the message was thrown</td><td></td></tr>
<tr class="bg2"><td><b>retry</b></td><td>INTEGER</td><td>NULL</td><td>retry_count of the job when the message was thrown (or NULL if no job)</td><td></td></tr>
<tr class="bg1"><td><b>status</b></td><td>ENUM('UNKNOWN','SPECIALIZATION','COMPILATION','CLAIMED','READY','PRE_CLEANUP','FETCH_INPUT','RUN','WRITE_OUTPUT','POST_CLEANUP','PASSED_ON')</td><td>'UNKNOWN'</td><td>of the job or worker when the message was thrown</td><td></td></tr>
<tr class="bg2"><td><b>msg</b></td><td>TEXT</td><td></td><td>string that contains the message</td><td></td></tr>
<tr class="bg1"><td><b>is_error</b></td><td>SMALLINT</td><td></td><td>binary flag</td><td></td></tr>
</table>
</div>
......
No preview for this file type
docs/hive_schema.png

249 KB | W: | H:

docs/hive_schema.png

262 KB | W: | H:

docs/hive_schema.png
docs/hive_schema.png
docs/hive_schema.png
docs/hive_schema.png
  • 2-up
  • Swipe
  • Onion skin
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment