View source for Visualization Subgroup (work in progress)

''Note: this page contains the unorganized notes taken by the '''Visualization''' [[Subgroups|subgroup]] during the [[Database_Interop_Hackathon|NESCent Hackathon on Database Interoperability]]. For a more finished summary of the group's activities, please go back to the main [[Visualization Subgroup|subgroup page]].

== Example XML URLs ==
These are URLs for NexML files that have been fully tested in PhyloWidget.

* http://dbhack1.googlecode.com/svn/trunk/visualization/www/data/ensembl.xml
* http://dbhack1.googlecode.com/svn/trunk/visualization/www/data/morphbank.xml

== Participants ==
* [[Greg]], [[Sheldon]], [[Katja]], [[Arlin]]

==Tasks List==
*[X] Describe / document standard "base level" decoration types for PhyloWidget to implement (i.e. node color, branch thickness, etc.)
**Implement these low-level decorations in PhyloWidget
**Done. -[[User:Gjuggler@gmail.com|Gjuggler@gmail.com]] 17:33, 10 March 2009 (EDT)
**Basic tree decoration UI added [[User:Mckays|Mckays]] 17:37, 12 March 2009 (EDT)
*[ ] Create "dummy" PhyloWS server (based on Sheldon's demo)
*[X] Create hooks in web app for remote tree-manipulation services
** Implement one or two of these
** Done by Greg, Sheldon, Katya
*[X] Implement NeXML input/output into PhyloWidget
**Done. -[[User:Gjuggler@gmail.com|Gjuggler@gmail.com]] 17:33, 10 March 2009 (EDT)
*[?] Fix up PhyloWidget stability issues
**Done. -[[User:Gjuggler@gmail.com|Gjuggler@gmail.com]] 17:33, 10 March 2009 (EDT)
**Still need a bit of work on graceful Applet destruction
*[X] Design UI for "discovering" annotations available for decoration
**Now available [[User:Mckays|Mckays]] 17:37, 12 March 2009 (EDT)
*[?] Listing concrete data use cases for visualization along the tree
** Well, some examples anyway

== Visualization use cases ==

=== Examples of tree visualization ===

* [http://www.phylowidget.org/ PhyloWidget]
** Input: Newick, NHX, NEXUS
** Output: Newick, NHX,
* [http://www.phycoweb.net/software/TreeGradients/ TreeGradients]
** Input: Newick, phyloXML
** Output: SVG
* [http://www.molevol.org/nexplorer Nexplorer]
** Input: NEXUS
** Output: PNG, NEXUS, PDF, PostScript
* [http://hymenoptera.tamu.edu/wiki/index.php/Main_Page mx]
* [http://mesquiteproject.org/mesquite/mesquite.html Mesquite]
** Input: NEXUS
** Output: NEXUS
* [http://macclade.org/macclade.html MacClade]
** Input: NEXUS
** Output: NEXUS

=== molecular use cases ===

# Branch Support Values
#* type of data: bootstrap values, Bayesian posteriors
#* token data file:
#* how its represented:
#** newick string with square-bracketed values: (rat,mouse)[0.95],((chimp, human)[0.99],squirrel)
#** tags in phyloxml or nexml
#* graphical metaphors:
#** number (in percent) next to descendant node
#** lower intensity of branch color
#* example implementations:
#** numbered nodes [http://www.molevol.org/nexplorer/cgi-bin/gdplot.pl?filename=PF00034_39.nex&dir=NEXUS&rand=5837&CGISESSID=671a2ebb8197165988d307be94f51ba0&show=Tree+and+Data&data=Protein&bootstrap=on&set=None&vertebrate=red&invertebrate=brown&plant=green&fungi=blue&protist=orange&width=5&colsize=5&spacing=1&submit1=Generate&output=png Nexplorer example]
#** branch color [http://www.phycoweb.net/software/TreeGradients/bootstrap_grays.jpg TreeGradients example]
# case name
#* type of data:
#* token data file:
#* how its represented:
#* graphical metaphors:
#* example implementations:
# case name
#* type of data:
#* token data file:
#* how its represented:
#* graphical metaphors:
#* example implementations:

=== morphological use cases ===
# trace character evolution on the tree (single character or few selected characters)
#* type of data:
#* token data file:
#* how its represented: numbered nodes, roll over to character information + link to anatomy ontology + standard view image representing character state
#* graphical metaphors: character states colored differently
#* example implementations:

# plot all characters on tree (reconstructed ancestral state)
#* type of data: character description
#* token data file:
#* how its represented: each node would be a roll over showing a window that displays ancestral states for each character at that particular node.
#* graphical metaphors: number node, colored node
#* example implementations:

# additional information: free text annotation
#* type of data: biogeographical, behavioral
#* token data file:
#* how its represented: each node or branch would have a graphic character representing user comment block, if multiple users working on one tree (collaborative environment) than each user may be represented by a different color.
#* graphical metaphors: color bars/lines
#* example implementations:

# additional information: url
#* type of data: link to external resource, specimen data, image data, voucher information for OTU, reference information
#* token data file:
#* how its represented: each node or branch would have a graphic character representing user comment block, if multiple users working on one tree (collaborative environment) than each user may be represented by a different color.
#* graphical metaphors: color bars/lines
#* example implementations: OTU links to specimen data information including voucher content.  Could be collections of images [morphbank image collection http://www.morphbank.net/myCollection/index.php?collectionId=221337]

== Problem definition ==

[[Arlin]]'s notes from day 1.

* categories of users to consider
*# user interested only in exploring or displaying data (no editing)
*# molecular evolution user:
*#* study unit is the sequence family (alignment, tree)
*#* problems of interest are things like detecting paralogy, function annotation, reconstructing chars
*# [[Tree of Life]] type
*#* study unit typically is a taxonomic group
*#* wants to discern species relationships
*# phylogeny producers
*#* wants to manipulate data, make supertrees, etc
*# morphologists
*#* study unit is a character complex, OTUs typically are species or higher
*#* wants to map characters and so on
* currently available technologies
*# [[PhyloWidget]] (java applet) - view trees, zoom, limited editing,
*# [[mx]] - view, annotate, solid back end
*# [[Nexplorer]] - view trees with char matrix, limited editing, Bio::NEXUS middleware with functions
*# ajax wrapper for phylowidget
* missing technologies
*# tree manipulation services (and interface standard, e.g,. [[PhyloWS]])
*# annotation language for "decorations", e.g., extend [[CDAO]]
*# rendering capability for graphic tokens (colors, thickened lines, pie charts, etc)
*# standards or conventions for  rendering
*# middleware for processing
*# UI design
* deliverable: web-based GUI to phylo resources using a standardized API
** use [[PhyloWidget]] as technology base
** support visualization of tree decoration
** support editing or annotation, including
*** free text annotation in key-value pair
*** controlled vocabulary referencing ontology such as CDAO
*** save info in database
*** session management
** other aspirations (not for this week)
*** plugin architecture

==Example files==
* [http://phenoscape.svn.sourceforge.net/viewvc/phenoscape/trunk/data/phenex-files/Tree%20Files/Vari_1995.nex?view=markup NEXUS tree file] and [http://phenoscape.svn.sourceforge.net/viewvc/phenoscape/trunk/data/phenex-files/Characiformes/Vari-etal-1995.xml?view=markup semantically marked-up NeXML character matrix] for Vari et al 1995 (from Phenoscape).

*  [http://seltmann.morphbank.net/dbhack1/ nexml parsed for otus, return images from morphbank]. More information on morphbank webservices found in [[Katjas_Notes]].

*  http://code.google.com/p/dbhack1/source/browse/trunk/mb_image_search/apoidea_tree_otu.xml  example tree file with otus



==Notes==
[[Sheldons_Notes]]

[[Gregs_Notes]]

[[Katjas_Notes]]

== Products ==
=== Summary ===
The outcome of the hackathon was the addition of new functionality to PhyloWidget and the development of a web-based application to facilitate database queries and user interaction.

[[Image:PhylowidgetWeb.png|border]]

=== Overview of Changes ===
;Phylowidget
:added NeXML I/O
:added remote web service deployment interface
:several new hooks for JavaScript interactions and callback methods
:enhanced storage of user-added annotations in NeXML

;PhyloWidget Web application
:exensive user-interface improvements
:added annotation viewer/editor
:added remote web service deployment interface

=== Demonstration Server ===
A development version can be browsed [http://iptol.iplantcollaborative.org/hackathon here]

=== Source Code ===
*[http://code.google.com/p/dbhack1/source/browse/#svn/trunk/visualization/www Browse the SVN repository]
*[http://iptol.iplantcollaborative.org/phylowidgetweb.tar.gz Download the tarball]

== Standup Reports ==

=== Wednesday ===

Given by [[Sheldon]], based on [[Dave]]'s notes.

* Good momentum
* [[Greg]] has [[NeXML]] IO for app
* added new functionality for tree decoration
* Sheldon on UI for that and editing
* [[Katja]] working on unambiguous ID resolution.
* [[PhyloWidget]] supports images on nodes.  Looking to generalize that.
* Switch from ad hoc to [[NeXML]] infrastructure.
* Deployable web servies.
* Nail down fundamentals of user interface today and then hook it up to real data.

=== Thursday ===

Given by [[Arlin]] and [[Greg]]; notes by [[Dave]].

* Creating [[NeXML]] files for different visualizations.
* Getting test framework working.
* Get prototype for demo today.
* Still holding off on the back end with hope this will be pluggable.
* Working on reading and writing (updated) [[NeXML]] files.