diff --git a/doc/MAC_G2.README b/doc/MAC_G2.README
deleted file mode 100755
index 593ad7e10fb0211a69bc9aeb371e61128c8429f0..0000000000000000000000000000000000000000
--- a/doc/MAC_G2.README
+++ /dev/null
@@ -1,87 +0,0 @@
-
-g2 install and autoconf on mac
-------------------------------
-
-Things we need to manage to get the g2 install to work on the mac.
-It is currently installed in ~zmap/prefix. But other people won't
-have this installed. This needs to be done. We can diable the
-feature in zmap, but that's not really a solution. In addition
-our build needs to check that it is installed and either error and
-go no further or warn and manage the features running without g2.
-
-g2 install
-----------
-
-This seems to be using an *old* version of autoconf.
-
-- configure.in needs renaming to configure.am
-- darwin needs an option in the switch statement.
-- Makefile.in needs renaming to Makefile.am
-- Makefile.am needs rewriting I fear.
-- ranlib needs running on the libg2.a
-
-I'll write some patches for this which will go in a
-patches directory.
-
-So all of the above were true, plus a few more glitches which needed ironing out.
-
-
-Step by Step
-------------
-
-Installing GD
--------------
-
-A little belatedly I found this which is roughly what I did.
-http://www.paginar.net/matias/articles/gd_x_howto.html
-
-I missed out a few bits, so here's what I did in case the omission
-causes issues.
-
-$prefix=/sw
-
-- zlib was already installed
-
-- install of libpng (see ~zmap/packages [cbi1] for tar.gz)
- This just installed using ./configure --prefix=$prefix; make; make install
-
-- install of libjpeg (see ~zmap/packages [cbi1] for tar.gz)
- This was slightly more tricky.
- ./configure --prefix=$prefix
- make all
- make install
- make install-lib
- ranlib $prefix/lib/libjpeg.a
-
-- install of libgd 2.0.33 (see ~zmap/packages [cbi1] for tar.gz)
- ln -s `which glibtool` libtool
- CFLAGS='-I/usr/X11R6/include/freetype2 -I/usr/X11R6/include/fontconfig' \
- ./configure --prefix=$prefix
- make
- make install
-
-
-Modifying G2
-------------
-
-Firstly to get anything to work.
-
-cd g2-0.70
-touch NEWS AUTHORS ChangeLog
-cp $ZMAP_CVS_DIR/src/bootstrap .
-./bootstrap
-
-rm -f configure.in
-configure.ac replaces this and is located in our cvs under g2
-
-rm -f Makefile.in
-Makefile.am replaces this and is also located in cvs under g2
-
-libtoolize
-./bootstrap
-configure needs to find gd, hence /sw/{include,lib}, and libiconv, hence /usr/lib
-CFLAGS=-I/sw/include LDFLAGS='-L/sw/lib -L/usr/lib' ./configure --prefix=$prefix
-make
-make install
-
-This doesn't install perl. When I learn how to do it I will.
diff --git a/doc/PERFORMANCE.README b/doc/PERFORMANCE.README
deleted file mode 100755
index e0a8f774d66801d7a4b3581797474bac21b3b2ba..0000000000000000000000000000000000000000
--- a/doc/PERFORMANCE.README
+++ /dev/null
@@ -1,478 +0,0 @@
-Summary of performance stuff....
-
-I started out thinking the memory allocator might be the problem but the memory/timing
-trace showed this not to be so (see [1] below).
-
-
-By interrupting zmap while parsing I realised it was spending a lot of time searching
-through g_datalist structs and I realised that:
-
-
-When adding a feature to a featureset during parsing we basically do:
-
-if (feature not in feature set)
- add feature to feature set
-
-
-this results in the g_datalist code doing two linear searches, one to see if the
-feature is there and then a second one to see if its there when it adds it....
-
-time to parse busy clone = 140s
-
-
-so the first change I made was to add a function to the g_datalist code that just adds
-the feature without checking whether its there, after all we've already done that....
-
-time to parse busy clone = 74s
-
-
-then I added a hash table to the featureset so we could use that to check whether we
-had the feature there instead of stupid g_datalist linear lookup...
-
-time to parse busy clone = 5s
-
-
-See [2] below for overall timings which are interesting.
-
-
-Now the two biggest delays are in getting the data from the server and in making the
-canvas items. We need to look at the latter at some time especially for splitting
-windows.
-
-
-
-
-[1] Memory/timing Output
-
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-../../../zmapView/zmapView.c:zMapViewConnect:266 Open connection - elapsed time: 1.2e-05
-../../../zmapThreads/zmapSlave.c:zmapNewThread:140 In thread, calling handler function - elapsed time: 0.003583
-thread_mem malloc
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-../../../zmapServer/acedb/acedbServer.c:getFeatures:433 In thread, getting features - elapsed time: 0.440321
-../../../zmapServer/acedb/acedbServer.c:sequenceRequest:707 In thread, about to ask for features - elapsed time: 0.440428
-../../../zmapServer/acedb/acedbServer.c:sequenceRequest:728 In thread, got features and about to parse into context - elapsed time: 7.5291
-../../../zmapServer/acedb/acedbServer.c:sequenceRequest:878 In thread, finished parsing features - elapsed time: 146.297
-../../../zmapServer/acedb/acedbServer.c:getFeatures:438 In thread, got features - elapsed time: 146.297
-../../../zmapThreads/zmapSlave.c:zmapNewThread:146 In thread, returned from handler function - elapsed time: 148.698
-../../../zmapView/zmapView.c:getFeatures:2040 Got Features from Thread - elapsed time: 148.751
-../../../zmapView/zmapView.c:mergeAndDrawContext:2003 Merged Features into context and about to display - elapsed time: 148.751
-../../../zmapWindow/zmapWindowDrawFeatures.c:zmapWindowDrawFeatures:211 About to create canvas features - elapsed time: 148.751
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-../../../zmapWindow/zmapWindowDrawFeatures.c:zmapWindowDrawFeatures:376 Finished creating canvas features - elapsed time: 160.058
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-malloc alloc: 1048
-
-
-
-[2] Timings for zmap.
-
-
-without any fixes:
-
-
-../../../zmapView/zmapView.c:zMapViewConnect:266 Open connection - elapsed time: 1.2e-05
-../../../zmapThreads/zmapSlave.c:zmapNewThread:140 In thread, calling handler function - elapsed time: 0.003583
-../../../zmapServer/acedb/acedbServer.c:getFeatures:433 In thread, getting features - elapsed time: 0.440321
-../../../zmapServer/acedb/acedbServer.c:sequenceRequest:707 In thread, about to ask for features - elapsed time: 0.440428
-../../../zmapServer/acedb/acedbServer.c:sequenceRequest:728 In thread, got features and about to parse into context - elapsed time: 7.5291
-../../../zmapServer/acedb/acedbServer.c:sequenceRequest:878 In thread, finished parsing features - elapsed time: 146.297
-../../../zmapServer/acedb/acedbServer.c:getFeatures:438 In thread, got features - elapsed time: 146.297
-../../../zmapThreads/zmapSlave.c:zmapNewThread:146 In thread, returned from handler function - elapsed time: 148.698
-../../../zmapView/zmapView.c:getFeatures:2040 Got Features from Thread - elapsed time: 148.751
-../../../zmapView/zmapView.c:mergeAndDrawContext:2003 Merged Features into context and about to display - elapsed time: 148.751
-../../../zmapWindow/zmapWindowDrawFeatures.c:zmapWindowDrawFeatures:211 About to create canvas features - elapsed time: 148.751
-../../../zmapWindow/zmapWindowDrawFeatures.c:zmapWindowDrawFeatures:376 Finished creating canvas features - elapsed time: 160.058
-
-
-with the g_datalist direct add code added:
-
-../../../zmapView/zmapView.c:zMapViewConnect:266 Open connection - elapsed time: 1.2e-05
-../../../zmapThreads/zmapSlave.c:zmapNewThread:140 In thread, calling handler function - elapsed time: 0.003591
-../../../zmapServer/acedb/acedbServer.c:getFeatures:433 In thread, getting features - elapsed time: 0.392993
-../../../zmapServer/acedb/acedbServer.c:sequenceRequest:707 In thread, about to ask for features - elapsed time: 0.393097
-../../../zmapServer/acedb/acedbServer.c:sequenceRequest:728 In thread, got features and about to parse into context - elapsed time: 7.58187
-../../../zmapServer/acedb/acedbServer.c:sequenceRequest:878 In thread, finished parsing features - elapsed time: 81.2551
-../../../zmapServer/acedb/acedbServer.c:getFeatures:438 In thread, got features - elapsed time: 81.2551
-../../../zmapThreads/zmapSlave.c:zmapNewThread:146 In thread, returned from handler function - elapsed time: 83.5875
-../../../zmapView/zmapView.c:getFeatures:2040 Got Features from Thread - elapsed time: 83.6691
-../../../zmapView/zmapView.c:mergeAndDrawContext:2003 Merged Features into context and about to display - elapsed time: 83.6691
-../../../zmapWindow/zmapWindowDrawFeatures.c:zmapWindowDrawFeatures:211 About to create canvas features - elapsed time: 83.6692
-../../../zmapWindow/zmapWindowDrawFeatures.c:zmapWindowDrawFeatures:376 Finished creating canvas features - elapsed time: 95.0206
-
-
-with the hash lookup added:
-
-../../../zmapView/zmapView.c:zMapViewConnect:266 Open connection - elapsed time: 1.2e-05
-../../../zmapThreads/zmapSlave.c:zmapNewThread:140 In thread, calling handler function - elapsed time: 0.003589
-../../../zmapServer/acedb/acedbServer.c:getFeatures:433 In thread, getting features - elapsed time: 0.40506
-../../../zmapServer/acedb/acedbServer.c:sequenceRequest:707 In thread, about to ask for features - elapsed time: 0.405164
-../../../zmapServer/acedb/acedbServer.c:sequenceRequest:728 In thread, got features and about to parse into context - elapsed time: 7.94696
-../../../zmapServer/acedb/acedbServer.c:sequenceRequest:878 In thread, finished parsing features - elapsed time: 12.3263
-../../../zmapServer/acedb/acedbServer.c:getFeatures:438 In thread, got features - elapsed time: 12.3263
-../../../zmapThreads/zmapSlave.c:zmapNewThread:146 In thread, returned from handler function - elapsed time: 14.8065
-../../../zmapView/zmapView.c:getFeatures:2040 Got Features from Thread - elapsed time: 14.8171
-../../../zmapView/zmapView.c:mergeAndDrawContext:2003 Merged Features into context and about to display - elapsed time: 14.8171
-../../../zmapWindow/zmapWindowDrawFeatures.c:zmapWindowDrawFeatures:211 About to create canvas features - elapsed time: 14.8172
-../../../zmapWindow/zmapWindowDrawFeatures.c:zmapWindowDrawFeatures:376 Finished creating canvas features - elapsed time: 26.1438
diff --git a/doc/column_config.README b/doc/column_config.README
deleted file mode 100755
index 3a7737a814721975ca55068ead4867b372abf257..0000000000000000000000000000000000000000
--- a/doc/column_config.README
+++ /dev/null
@@ -1,46 +0,0 @@
-Column Ordering
----------------
-
-There are two column orderings required for zmap, the columns in the main zmap window and the
-column orderings in the navigator window. These are specified in the ZMap configuration file in
-the "source" stanza:
-
-
-source
-{
-featuresets = "Locus \"3 Frame\" \"3 Frame Translation\" vertebrate_mRNA Assembly_Tags etc."
-navigator = "Genomic_canonical Locus Scale"
-}
-
-(Note that feature set names can have embedded blanks)
-
-These lists not only specify the order of columns but also control which feature sets will be
-displayed, if the feature set (or its Column_group) is not in this list then the feature will not
-be displayed.
-
-
-Other Column settings
----------------------
-
-acedb did not allow configuration of attributes such as "between column" spacing, we could
-add this to the acedb database in the form of some kind of "meta style" but probably its
-better to simply specify this in the configuration file.
-
-Currently we have:
-
-"canvas_maxsize" maximum size in pixels of canvas window
-"canvas_maxbases" <not used>
-"keep_empty_columns" maintain empty columns to keep even spacing on screen between blocks
-"display_forward_coords" when specified means coords for reverse complement view are -ve of forward coords
-"show_3_frame_reverse" when specified means 3 frame cols are displayed on reverse strand as well
-
-spacing:
-
-"align_spacing" these all control the horizontal space between various parts of zmap,
-"block_spacing" probably column spacing is the most relevant.
-"strand_spacing"
-"column_spacing"
-"feature_spacing"
-
-"feature_line_width" controls default thickness of lines used for feature display.
-
diff --git a/doc/config.README b/doc/config.README
deleted file mode 100644
index 523a345dc51fecff91c6fc5a8ed431d984c6af7f..0000000000000000000000000000000000000000
--- a/doc/config.README
+++ /dev/null
@@ -1,105 +0,0 @@
-zmapConf - a tidied up version of zmapConfig
-mgh 09 Dec 2009
-
-Refer to ZMap/doc/config.txt for some discussion of the previous code which was distributed amongst the zmapConfig/ and zmapUtils/ and zmapFeature/ directories, including some not very obvious files (eg zmapStyleUtils.c), and also included the previous version of code which was not used.
-
-For this version all the config handling code will reside in zmapConf and documentation regarding use will be updated and put on the developer web site.
-
-Here's a summary of what and how:
-
-1) Reading Files
-
-The user may specify a config file by a command line argument but if not a default file ~/.ZMap/ZMap will be used. ZMap may also refer to system and application specific config files if installed (/etc/ZMap and $ZMAP_HOME/ZMap). If so then the data in the system and application files will take precedence
-
-The ZMap file may contain stanzas and keys as defined by the GLib GKeyFile module. Multiple instances of a stanze will be combined and if multiple instances of a key exist thent last one will take precedence.
-
-Any number of extra config files may be refered to by keys and any nuumber of these may be loaded on request by the application - by default only the main config file will be processed.by ZMap's main thread.
-
-In cases where multiple instances if a single stanza type are required (eg for multiple data sources) then either a list of stanzas will be specified by a key, or the stanza names will be prefixed by the stanza type (eg [source:wublastx]). ZMap will get a list of all stanzas and process each appropriately.
-
-A name key will be added to all stanzas if not present.
-
-
-2) Writing files (and editing)
-
-Currently only the [Blixem] stanza needs to be updated by ZMap, but if so the entire ZMap config file will be written. Note that this applies to the 'user config' only, not the system or application files (if present).
-
-There are modules in the code that edit a memory copy of config data. Although the use the config module they are not part of it and if included in config code will be moved elsewhere. (Not relevant)
-
-
-3) Stanzas and Keys
-
-Here it is assumed that the reader is familar with the existing scheme and some modifications are proposed:
-
-In the [source] type stanzas options are available to define featuresets and styles. Coincidentally at present these are identical and style names could be defaulted to be the same as the featureset name - this makes sense as when a source provides featuresets data it would be expected to supply styles as part of this.
-
-Styles may be defined in a config file which can be specified in [ZMap] with the key styles-file=filename, and only one will be allowed.
-Individual source stanzas will not have a styles-file key.
-
-Styles may be defined in a source stanza as at present - it is inteneded that DAS servers will be able to request styles in future - but sources are not required to process this. ACEDB servers have styles defined by thier featuresets and do not use styles config. File and Pipe servers do not use styles config as they assume that featuresets are displayed using a style of the same name, wever they will honour the list of styles if given.
-
-Options to specify ambiguous stanzas have been removed eg [sources], [source] etc.
-Sources must be listed in [ZMap] sources=one ; two ;.... and the individual sources defined in a stanza each.
-In terms of policy [source] is no longer allowed ... but the code is still there, one [source] stanza will still work.
-
-
-
-4) Application interface
-
-Each module that uses configuration data will read in the config file(s) and create its own copy of config data each time that it initialises - for example creating a new window will result in up to date setting beings used for that window. Or the main thread may create new data sources when a new ZMap is created (any existing ZMap will continue unchanged).
-
-Application code can choose to read in extra files (specified by a config key or hard-coded), and this function will be provided by the zmapConfig module. Regardless of how many files are read for any given config data object all files and stanzas will be merged into a single interface.
-
-The files themselves are read into a blank GKeyFile data structure and stanza names and key values accessed by querying this GLib interface. Each stanza has an associated data structure defined by ZMap which is initialised by the config code and key data overlayed onto this.
-After reading in config file and creating these data structures the GKeyFile date may be freed.
-
-Key and stanza names/ strings are defined in a single file: include/ZMap/zmapConfigStrings.h. The file ZMapConfigLoader .c has all the code for constructing stanzas. It was intended to scatter stanza specific data to modules that used it but this preoved impractical due to the need for several different modules to access the same stanzas.
-
-All the config handling code is now in zmapConfig/ and there are 15 source files the use it (grep for ConfigIni.h to find them).
-
-
-5) User guide
-
-(This needs to be updated)
-
-
-6) Other features
-
-NB: Styles data - styles can refer to parent styles to an unlimited depth, but this feature is limited to styles defined in a single file. A child inherits all the parent's data and the overlays its own. Multiple definitions of a style result in the original one being used.
--- not checked --
-
-
-7) Technical details
-
--- Source files, config data structures and related functions --
-
-zmapConfigDir.c finding the config files, internal only,
- well nearly: used by AppWindow, Logging, and View (for iffed-out code)
-
-zmapConfigFile.c reading and writing files, internal only
-zmapConfigKey.c get and set values, internal only
-
-zmapConfigIni.c the main external interface, read files, find stanzas, get key values
-zmapConfigLoader.c part 2 of the external interface
- initialise stanza structures. Read groups of stanzas (sources and styles).
-
-headers:
-include/ZMap/zmapConfigIni.h what application modules need most often
-include/ZMap/zmapConfigStanzaStructs.h
-include/ZMap/zmapConfigStrings.h
-include/ZMap/zmapConfigDir.h
-zmapConfig/zmapConfigDir_P.h private header for ConfigDir
-zmapConfig/zmapConfigIni_P.h private header for ConfigIni
-
-
-Internal to the zmapConfig module there is a ZMapConfigIni struct which is simply a list of GKeyFiles. Functions which deal with this are in the files zmapConfigFile.c and zmapConfigKey.cand are generally named -ConfigIniThing();
-
-External (ie visible from outside) to the zmapConfig module there is a ZMapConfigIniContext struct which includes a ZMapConfigIni and a few other items. Pretty much all of the global functions in zmapConfig take an -IniContext structure, pass on the -Ini struct and ignore all the rest,
-
-zmapConfigIni.c and zmapConfigLoader.c hold the external functions, External functions are called -IniContextThing()
-zmapConfigDir.c was intended as internal but turn out to have some functions called from startup code.
-
-
-
-
-
diff --git a/doc/config.txt b/doc/config.txt
deleted file mode 100644
index 9880aeaf53a95f7907d8973772fc7ec4a625375d..0000000000000000000000000000000000000000
--- a/doc/config.txt
+++ /dev/null
@@ -1,172 +0,0 @@
-ZMap Config
-mgh Dec 2009
-
--- A slightly pedantic look at a ZMap module --
-
-Please refer also to src/zmapConfig/README.
-
-What does ZMap Config do?
-
-- read in any number of text files and combines these into a single flat resource (extra config files may be specified in a config file)
-- reads data organised into un-nested stanzas
-- reads tokens of different types: integer, float, string, arrays, booleans...
-- optionally several different stanzas of the same name may exist
-
-- writes user config to store session data? (only for Blixem)
-- maintains a version history of user data? (not really)
-
-- specifies an internal data format for the data and provides functions to access it (using GLib)
-- defines (via data structures) what data and options may be specified by the user.
-
-- provides a default directory and main config file and allows alternate ones to be specified
-
-Some of the comments are clearly out of date and it looks like the functional spec has migrated.
-
-
-How much config data and code is there?
-
-In a simple installation (development) in the default ZMap file there are 5 types of stanza: [ZMap], [source], [logging], [blixen], [window], (76 lines of data) and in another file there are 136 [style] stanzas (specifying display formats for columns - about 1000 lines of data)
-
-At a rough guess there are currently 3500 lines of code (not including headers) to drive config files and 1000+ lines using it. There are 75 files out of 181 that include the string 'config'.
-
-
-Is it thread safe?
-
-Arguably there is no need as all the data can be read in on program startup and stored in memory. Threads can be made to regard config as read-only, and the master thread (the view) can handle all user changable data. There may be an argument for threads maintaining thier own config files, but given that the point of having threads is to have more than one there are obvious security issues.
-
-
-Files
-
-zmapConfig.c "no longer used"
-zmapConfigRead.c not used
-zmapConfigWrite.c one function, not called
-zmapConfigGet.c returns stanzas from stored data, not used
-zmapConfigUtils.c not used
-
-zmapConfigDir finds where the config file(s) are
-zmapConfigIni.c -- this does a lot of the work -- ** appears to be 2 files pasted together **
-zmapConfigLoader.c -- this does a lot of the work -- ** in the zmapUtils directory **
-
-see also:
-zmapFileUtils.c
-zmapStyleUtils.c application code doing config functions
-
-
-
-The Config File Menagerie
-
-All realistic locations appear to have been catered for:
-
-Hard coded (defaults?)
-System
-Application Home
-User Home
-User supplied
-
-There is also a 'default_dir' which is not used.
-
-There are functions that ultimately call zMapMakeUserConfig() (which writes an empty file with a comment header) in zmapView.c, but both of these are iffed out with comments like THIS_NEEDS_REDOING and NOT_NEEDED_ATM. (addAlignments() and getSequenceServers()C).
-
-
-
-How it works: (yes really)
-
--- Reading the Data --
-zmapConfigIni.c/zMapConfigIniContextCreate() calls zmapConfigLoader.c/zmapConfigIniContextProvide() and then zmapConfigIniR.c/zMapConfigIniReadAll() which will try to read all the files and then construct a merged (overlayed) version.
-'All the files' being system, application and user. A complete set of stanzas is created in memory and overlayed with data from the files, although it is not clear how defaults are handled: these could be set up as static data but are not.
-
--- Merging Groups and Files --
-GKeyFile will combine multiple stanzas with the same name into one; if multiple copies of the same key exist the last one wins.
-Multiple files get overlayed by the zmapConfigIni code.
-
-There are five key files recognised:
- files[0] = config->buffer_key_file; (not used - from zmapConfigIni days)
- files[1] = config->extra_key_file; (not used - from zmapConfigIni days - but see below)
- files[2] = config->user_key_file;
- files[3] = config->zmap_key_file;
- files[4] = config->sys_key_file;
-
-Currently the code is a little confused and 3 then 4 are prioritised, then 0,1,2 in that order, ie a user file setting will not overwrite a system file setting; this appears to be based on the theory that only keys that must not be changed will appear in these files. There is another function that checks for stanza existance which operates in the opposite order. If more than one extra file is read (or the same one mentioned more than once) then there is a memory leak.
-
-Additionally, extra files can be read in by request (eg the styles config, called from zmapServerProtocolHandler.c/getStylesFromFile() via zmapStylesUtils.c/zMapFeatureTypeGetFromFile(), and this is stored in config->extra_key_file
-
--- Saving Config Data --
-Currently used to remember Blixem setup, the entire user file is written using GKeyFile functions. Blixem also appears to have its own config file, and the name of this is defined inthe ZMap config but not read by ZMap.
-
--- Stanzas: Allowed Names and Keys --
-In the ZMap config file there are a few stanzas that are allowed and these are created in zMapConfigIniProvideContext() with the full set of allowed keys. Stanzas that can be duplicated (eg [source] and [style]) must be named uniquely or else will overwrite - these are processed directly by application code.
-
-
-
-The Plan
-
-- tighten up the design and document it
- define merging rules
- define the use of extra files and bring this into the Config module not the application code
- change ZMapConfigIniStruct to be a list of key files that can be extended as files get read,
- priority being the order of files in the list
- take Config file use out of threads - read all data in the main thread (GUI)
- implement reading of 'extra' config files in the top level files (system, application, user) in the stanzas that need to use this
- add styles-file=XXX into [ZMap] and make it a list like sources and styles. (but keep it as an option in [source])
- a default user file to be created by a command line argument option
-
-- unused functions to be deleted from the source
-
-- zmapConfigLoader.c to be moved to zmapConfig from zmapUtils
-- zmapServerProtocolHandler.c to call functions that parse stored data rather than reading files
-- any other application code handling config directly (eg in zmapView) to be adjusted
-
-- a new set of files created (eg zmapConf*) and the old once replaced to avoid confusion, and to allow old code to be restored from the CVS if necessary. Old (especially unused) files to be deleted from the CVS.
-
-- function names and structures, data to be changed (eg zmMapConf*) to ensure that old code does not escape inspection
-
-- doxygen and other documentation put online
-
-
-Questions
-
-Why does ZMAP store Blixem config when it has it's own file?
-Can Blixem be used without Otterlace (other than for development)?
-Is there milage in removing the write config file functions, or creating a seperate file for each writable group of data ? (more secure)
-
-
-
-Time and Effort Estimates
-
-- writing the functional spec (merging rules, handling of files, application interface, user guide) 2 hrs
-mainly expanding on the above
-- writing a technical spec: files and function definitions, data structures, data in and data out 2 hrs
-very brief, again expanding on the above
-** see zmapConf/README for further details **
----** see zmapConf/README for further details ------------------------------------------------------- OK
-
-- create new directories and files, cut and paste functions from old code to new 4 hrs ?
-~2500 lines of code
----Builds + runs ok, some issues on source organisation --------------------------------------------- OK
-
-
-- implement extra files and merging rules, modify stanza handling to match 2 hrs
---- only 1 buffer config and only 1 extra file config exists, so keep existing hard coded pointers --
----directory structure no cleaner, only zmapConfig/ is used------------------------------------------ OK
-
-
-- create default file on command line arg 2 hrs (not done)
-
-- renaming types and functions to force editing and clean compile relating to config code 1 day
-75 files include the string 'config', 18 include 'configinicontext' (260 instances)
-317 instances of 'configini' in 18 files - this is realistic. Most will require no change.
----most functions not changed, external references are now relatively clean-------------------------- OK
-
-- doxygen comments (currently not there at all) 1 day
-
-- OPTIONAL clean compiles of all affected source files where no difficult issues arise 1 day
-
-- operating the CVS
-add new files and directories and deleting old ones and comitting the source 1 hr
------------------------------------------------------------------------------------------------------ OK
-
-- running ValGrind over the code and fixing ConfigIni related issues 1 day
-NB this is fixing existing bugs, not updating the module
-
-
-In rough terms 3 days to do the work, 1 to write it up, plus 1 day to fix old bugs w/ ValGrind.
diff --git a/doc/pipeServer_config.txt b/doc/pipeServer_config.txt
deleted file mode 100644
index 19527b6a7c25fc2209d678ad76b5d7aad5235f4e..0000000000000000000000000000000000000000
--- a/doc/pipeServer_config.txt
+++ /dev/null
@@ -1,87 +0,0 @@
-Scripting interface and configuration for the ZMap pipeServer module
-mgh 27 Nov 2009
-
-The pipeServer module is based on the existing fileServer and is configured using a [source] stanza in the ZMap config file for each pipe. Scripts are to be installed locally with ZMap and the directory itself will be identified in the [ZMap] stanza in ~/ZMap/ZMap.
-eg:
-
-[ZMap]
-script-dir=/nfs/users/nfs_m/mh17/ZMap/scripts
-# Of course these could be stored centrally somewhere if desired.
-
-# Each data source must be referenced in [ZMap] by listing the source stanzas like this:
-sources = acedb ; other-feed ; yet_another_one
-
-# Columns are displayed in the order given: in the example above all the acedb features first then the 'other_feed' etc.
-
-
-# If styles are noe defined via ACEDB the a file must be given in the ZMap stanza (not the source stanza) eg:
-stylesfile = /nfs/users/nfs_m/mh17/zmap/styles/ZMap.b0250.file.styles
-
-
-When configured, ZMap will request data from each source in parallel, hopefully speeding things up a lot. Each script will obtain and send the data 'somehow'. They will replace the existing mechanism of Otterlace retreiving the data sequentially and adding to ACEDB on startup.
-
-
-In each source stanza (one must exist for each data source) the syntax is the same as for existing file:// and acedb:// sources, but specifically for pipe:// sources we interpret the configuration as follows:
-
-URL's take the form
-
- <scheme>://[user][:password]@<host>[:port]/[url-path][;typecode][?query][#fragment]
-
-and:
- <scheme> will be 'pipe'
-
- user:password@host are not used and if present are ignored
-
- port is not used and if present will be ignored
-
- url-path is the path of the script.
- Note that according to http://rfc.net/rfc1738.html a single leading '/' signifies a relative path and two signifies absolute. We will interpret relative paths as relative to the ZMap scripts directory.
-
- typecode is not used and will be ignored if present
-
- query will be expanded into a normal argv vector
-
- fragment is not used and will be ignored
-
-
-Typically we expect a pipe:// data source to have only one (or very few) feature sets, as a major design aim is to exploit concurrent operation. Other configuration parameters will operate as normal (eg 'sequence=true' (which can only appear in one source) and 'navigator_sets=xxx,yyyy').
-
-Here is an example for a test script that simple outputs an existing GFF file.
-
-[source]
-url = pipe://getgff.pl?file=b0250_curated.gff
-featuresets = curated_features ; curated ; genomic_canonical
-styles = curated_features ; curated ; genomic_canonical
-
-
-
-A more realistic one with an absolute path... (but needs featuresets and styles and stylesfile specifying)
-[source]
-url=pipe:///software/anacode/bin/get_genes.pl?dataset=human&name=1&analysis=ccds_gene&end=161655109&csver=Otter&cs=chromosome&type=chr1-14&metakey=ens_livemirror_ccds_db&start=161542637&featuresets=CCDS:Coding;CCDS:Transcript
-
-
-
-
-Script operation: some rules
-
-A script must start with #!<program> or else it will not be exec'd. (Assuming Linux)
-
-A script may obtain data in any way it likes but must output valid GFF data and nothing else on STDOUT (but anything is valid in a comment).
-
-Brief error messages may be output to STDERR and these will be appended to the zmap log. STDERR output is intended only to alert users of some failure (eg 'warning not all data found' or 'cannot connect to database') and not as a detailed log of script activity - if this is needed then the script should maintain its own log file. A warning message will be presented to the user, consisting of the last line in STDERR and hopefully this will be enough to explain the situation with resorting to log files.
-Regardless of whether an error message is sent ZMap will attempt to use the GFF data provided.
-ZMap will probably read STDERR after STDOUT is closed, and only if some error is encountered.
-
-Arguments will be given in the format key=value with no preceeding dashes, these will be as extracted from the server query string. (If people care about this we could change it...)
-
-Extra arguments may be added subject to implementation:
-
-zmap_start=zmap_start_coord // in zmap coordinates not bases (if configured)
-zmap_end=zmap_end_coord
-wait=9 // delay some seconds before sending data (can be given in the query string, main use is for testing)
-
-
-
-
-
-
diff --git a/doc/styles.README b/doc/styles.README
deleted file mode 100755
index 09ab3e41f994e2d1066dd98cf30dd3a2c16064ff..0000000000000000000000000000000000000000
--- a/doc/styles.README
+++ /dev/null
@@ -1,2 +0,0 @@
-See ../web/user_doc/styles.shtml
-@ http://wwwdev.sanger.ac.uk/Software/analysis/ZMap/user_doc/styles.shtml