minutes from meeting

ZMAP_LACE_PROJECT/zmap_lace.2008_01_10

+==============================================================================
+ZMap/Otterlace Development
+
+
+Date:  Thursday 10th January 2008
+
+Attendees: rds, eah, jgrg, kj2, st3, jla1, lw2
+
+
+1) otterlace + zmap progress
+============================
+
+
+High priority
+-------------
+
+
+0/ ZMap performance - contd.....
+
+Although performance is still proving to be an issue in some cases, it
+is felt  that incorporating 'missing'  features is a  higher priority.
+The plan is  to remove xace from the otterlace  environment as soon as
+possible, so features that  are absolutely essential to the annotators
+working should be addressed before the issue of performance.
+
+There is still a problem with the pipeline and pathological conditions
+that produce huge numbers of essentially useless homologies. jgrg is
+thinking about how to do this. 
+
+edgrif investigating two possibilities for improving performance:
+
+	- make sgifaceserver stream data rather than batch it up, would
+	  save a lot of memory.
+
+	- deferred loading, only load features when needed and load in
+	  zone requested by user....design done...now need to implement.
+
+Have found X performance tools, very comprehensive. Plan is to run them
+on my machine as a benchmark and compare. Will produce noddy script that
+records the machines configuration, runs the X performance tool and produces
+a comparison with the benchmark. This way we can profile anyones machine
+who is having problems with performance.
+
+
+
+1/ New zmap styles *************
+
+edgrif has now implemented this (with inheritance), jgrg needs
+to decide when to do the big swop. jgrg said with the new schema
+work now almost done he is in a position to do this.
+
+The lack of  zmap styles is holding up  development and implementation
+of certain zmap features.  This makes this a high priority item.
+
+Should be done this week. edgrif  & jgrg to discuss how to turn styles
+"on" in zmap.
+
+2/ Display of otter information
+
+In particular the Feature  Details dialog/replicating some of treeview
+(ticket #49520).  Mark provided  a very  good mock up  of the  way the
+dialog should look which edgrif is working to.  
+
+Otter ids are now displayed. edgrif is reworking the feature display to match
+requirements from Mark who coordinated Havana input on this. ZMap code should be
+done this week and should include:
+
+lw2 also asked about DE line information display, edgrif asked lw2 to collect
+together requirements for information to be displayed. jla1 to chase up.
+
+39329:  PFAM info
+Possible to show a description associated with Halfwise (Pfam) objects in Zmap?
+Currently in lace we see the domain description as well as the pfam accession number
+(EAH) jgrg will pass this information over to zmap, in particular Ensembl URLs will
+be passed over.
+
+BLAST evidence info:
+Would it be possible to see what organism a piece of evidence belongs to, without
+having to pfetch each one individually? Also no info (field Title in Fmap) on EST/mRNA
+hits unless you pfetch (RJK, AF2, MT4) We need species and taxon data but some of
+this needs new database fields (jgrg to implement).
+
+st3 raised a user request to add the author field into this display.
+
+3/ Clone summary info/Automating DE line creation.
+
+eah raised  the point that  certain summary information for  clones is
+not available when using zmap.  e.g. the number of CpG islands.  These
+are  used for  the  authoring of  DE  lines.  There  is  a script  for
+automating this  which kj2 wrote  for zebrafish, but it's  specific to
+zebrafish and  requires running  on the command  line. jgrg  said this
+could be  integrated into  the clone editing  window in lace.  jgrg to
+action.
+
+
+4/ CDS translation
+
+From Jane
+
+Showing the  peptide translation  of an object  next to the  object in
+zmap with the residue numbers next to the exon boundaries is still not
+working.  It took me about 5  times as long to check my object without
+this functionality.
+
+Will be  in next  build which  is early next  week. Actually  it's not
+quite finished as it's currently quite unstable.
+
+rds is working on this. the code will depend on zmap styles to work.
+
+
+5/ parsing names with embedded spaces...
+
+This is causing a problem saving EUCOMM Exons.
+
+jgrg said there is a problem with zmap parsing gff where there are feature names
+with embedded spaces, edgrif to fix this asap. Also, the text following the object
+name needs to be displayed in the info. line when the feature is selected.
+
+6/ Consistency Test using ZMap
+
+jla1 said there will be a test in February using zmap, not xace. edgrif said he
+and Roy will make sure there is a stable version for then.
+
+jla1 said this would be only possible when points 1, 2 & 4 have been actioned.
+
+
+
+Medium priority
+---------------
+
+0/ 5'and 3' EST read pairs
+
+we need these to be marked in zmap as in acedb, requires new tags in database in
+the same way as in worm database.
+
+
+1/ locator column: zmap needs a locator column as per fmap which could be
+used to display dna and peptide search results and other information.
+
+
+2/ Annotation needs to have a history across assembly changes.
+
+kj2  requested that  when an  assembly changes  the objects  which get
+transferred  should have a  history as  to what  they were  (otter id)
+previously.  lw2 remarked that it's  possible to search for the old id
+in the lace  interface, but impossible to retrieve  the old object and
+display it.  jgrg  acknowledged there is a bug  in the lace searching,
+but  that   providing  the   history  would  be   difficult.   Further
+discussion/thought on how to implement this is needed.
+
+jgrg said that this should be more possible with the new ensembl schema.
+
+
+3/ clone overlap display
+
+lw2 said users need to see more clone overlap data, jgrg has provided a
+suggestion for how this might be displayed.
+
+They would also like to be able to list features by clone rather than just
+by hit name or whatever.
+
+They would also like to see the clone name in lists of hsps....
+
+
+
+4/ removing evidence already used
+
+*** TOP PRIORITY ***
+
+annotators would like to be able to remove from display homologies that
+have already been used to annotate variants etc. Does this need to be
+persistent in the database in some way ?? edgrif & jgrg will get
+together to arrange this via styles so it can persist in a natural way
+in the database.
+
+**24526: Showing which evidence has been used
+Differential coloring of matches that have been used already as evidence
+for a transcript
+
+
+5/ Multiple alignments
+
+multiple alignments: edgrif is about a third of the way through implementing a
+more general way of displaying arbitrary blocks.  This will become a high
+priority item as we move to haplotypes etc.
+
+th said this would be needed soon so it should be moved up the priority list.
+
+
+6/ pfetch proxy
+
+jgrg has provided pseudo code, rds to implement.
+
+
+7/ Interface issues:
+
+
+extending marked region:
+
+jla1,  eah and lw2  said users  would like  some way  of interactively
+extending the marked area. edgrif  to look at this perhaps using mouse
+action  over the  marked area.   eah also  requested that  this should
+alter the bump column so that evidence that has been hidden, as it did
+not overlap, gets shown.
+
+
+jla1 and lw2 said they would like the marked area to be less obvious an also to
+be a "greying" out rather than blue. edgrif to implement.
+
+jla1 said she would like to be able to click on an exon and see evidence (and
+transcripts ?) with the same splice be highlighted.
+
+
+Zooming set up:
+It would be very useful for large genes if the evidence, ensembl objects etc. did not
+disappear when you zoom out.  This happens faster in Zmap than in Fmap, but the havana
+objects do not disappear in either case. (MMS)
+
+edgrif explained that we need the new styles to fix this.
+
+
+lw2 said the sensitivity of  the lasso is still too sensitive.  edgrif
+has decreased this so that the lasso must be dragged >20 pixels.  This
+can be changed, but we need to have a consensus on this.
+
+Would it be possible to have an undo or back (like on a web browser) button? (AF2)
+
+rds said that this has been  provided to enable stepping back 1 event,
+which can either be a mark, unmark, or zoom operation.
+
+
+8/ There was a discussion about how much a user should be able to
+configure. It was agreed that they should be able to configure which columns
+are initially hidden. They should also be able to "save" the current settings
+to set this up and to be able to restore the system or there currently saved
+defaults.
+
+edgrif has implemented code for this but now need eah and lw2 to report back with
+list of what users would like to be able to configure.
+
+jla1 said that configuration is required more at the group or DB level, we can
+already do this via lace setting up zmaps configuration files.
+
+
+
+9/ Future stuff
+
+edgrif said he would like annotators to start thinking/reporting two things:
+
+- repetitive tasks that could be automated.
+
+- new ways to highlight/select data to help build/annotate transcripts.
+
+
+
+
+Low priority
+------------
+
+
+1/ jgrg suggested that short cuts should be given on menus/mouse-over popups
+to remind the user that they exist...edgrif to do this.
+
+
+2/ alternative translations: edgrif about half way through code to do this.
+
+edgrif is doing this as part of the protein search code since this code
+does translations itself. edgrif will talk to jgrg about how alternative
+genetic codes can be specified with acedb.
+
+We need a test database for this. jgrg said this would come soon.
+
+
+3/ loutre schema and data representation changes
+
+We need some way to make sure st3 knows about changes anacode are making.
+jgrg and st3 to communicate more over this.
+
+
+
+2) Back To The Future
+=====================
+
+
+Leo's transcript display
+------------------------
+
+There was a discussion about good ways to interact with
+alignments/transcripts for deletion and other actions. Leo and James have
+already worked out a lot of this with Leos transcript viewer. edgrif will get
+the rules from Leo and James and then we can discuss them and decide which to
+implement. edgrif suggested that it might be good to have a separate window
+but jgrg said he thought it could be done within existing zmap by adding
+some new features.
+
+
+A new canvas
+------------
+
+rds has been looking at alternative canvas implementations which offer an MVC
+model. He has managed to get goocanvas developers to fix some bugs and make
+some changes to support our needs.
+
+the goocanvas MVC model will mean we do not have to copy data to split windows
+meaning greatly reduced memory usage.
+
+the goocanvas will cope automatically with the X Windows window size limit, this
+combined with changes in the gtk scrolling model means we will be able to do away
+with having two scroll bars.
+
+We plan to introduce this in the Autumn when it has become part of gtk
+as _the_ gtk canvas.
+
+
+
+Changing the Assembly
+---------------------
+
+th made the good but frightening point that the world of annotation has
+changed to one where the annotator may well be making changes to the
+underlying assembly, he said that ensembl supports this and lace/zmap will
+need to. The consequences feel alarming.....
+
+edgrif and rds to think about this: would it mean that zmap would have to
+change the position of features on the fly as the user edits the
+assembly...some tricky pathological cases here....
+
+We will need smapping support in zmap to do this kind of thing. edgrif has
+started work on an autoconf'd package to do this.
+
+
+Builds
+------
+
+rds has done the universal binary builds. So we can provide hand builds of them
+but still need to incorporate the system into our overnight build procedure.
+A labour of love not helped by fairly poor docs from Apple. But it is now
+working.
+
+update has been incorporated into  overnight builds, but we still have
+library  clashes  with those  in  the  curernt  otterlace (xace  only)
+distribution.  James and I are working to resolve these.
+
+jgrg working on mac distribution to incorporate zmap.
+
+
+Blixem/pfetch
+--------------
+
+- blixem: dna searching is NOT DONE, edgrif to expedite. Also protein searches
+will be added.
+
+
+Turning off xace
+----------------
+
+The  plan is  to turn  off xace  in the  otterlace client  as  soon as
+possible.  This  requires actioning of  1, 2 and 4.   Inevitably there
+will be a  need to either quickly switch back or  have a version which
+still incorporates xace. jgrg to think how to do this.
+
+How many external users is this going to effect?  Do they need training?
+
+
+2) Other matters
+================
+
+kj2 raised a question about how alignments were done for otter, jgrg said
+using BLAST and est2genome. Kerstin said that much more useful alignments
+could be made using BLAT or exonerate or ??? In particular, other alignment
+methods join up their matches where they are obviously "perfect" overall
+alignments (e.g. to consecutive exons) and cope with alignments where they go
+over clone boundaries.
+
+Mustapha is working on this.
+
+
+- jgrg,  st3  &  jla1  discussed  the upcoming  Vega  (mouse)  release
+  including  which schema version  and assembly  version it  should be
+  built with. jla1 asked about updating to assembly NCBI37.
+
+
+3) Next Meeting
+===============
+
+
+
+
+==============================================================================