News flash: the 4th meeting of the working group will be a Database_Interop_Hackathon that takes on some of the interoperability challenges described below.
The goal of improving interoperability is not served merely by developing artefacts such as CDAO and nexml. In order to improve interoperability, these artefacts must be used in the research community and, ideally, they must become popular. Thus, to achieve its goals, the working group wishes to promulgate its artefacts, and in particular, we wish to develop a strategy to reach a "critical mass" in which there are sufficiently many operations and resources that rely on an artefact to ensure that the self-interested researcher (or the lazy programmer) embraces the artefact simply because its useful and not just for some intangible long-term benefit of "interoperability".
Our main focus is on the "carrots" part of a "carrots and sticks" strategy, but first we should explain the "sticks" part. Journals that serve the evolution research community have indicated a desire and a willingness to impose data archiving requirements on authors as soon as the technology becomes feasible (Todd Vision, pers. comm.; see Dryad project); also, the research community has called for a minimal reporting standard, MIAPA. These two aspects of interoperability go together: the repository goal favors a minimal reporting standard such as MIAPA to ensure that archived data are re-usable. Furthermore, these two aspects of interoperability go together with a third aspect, which is a file format that is MIAPA-compliant and that is accepted by archives. The "stick" part of our nexml-CDAO strategy is that we intend to support MIAPA, which means that some users may resort to using nexml-CDAO if it is the most convenient way to achieve MIAPA-compliance or to archive data.
The tentatively named CarrotBase project is intended to provide a more positive incentive, i.e., to represent the "carrots" part of the strategy. When we constituted the working group in 2006, we clearly were targeting developers of phylogenetics applications software. This might have been the best way to start developing the technology for interoperability-- artefacts such as nexml and CDAO-- but now we are faced with a different problem of promulgating these technologies. Targeting applications does not seem like the best way to generate carrots for users. Targetting data resources might be more effective.
List of data resources to target
For more information, and analysis of target potential, see Data Resources.
- resources for systematics
- resources focused on sequence families
- TreeBaseII has the kind of granular schema that would be a good challenge to try to accommodate using cdao.
- pPOD (not really a data resource, its a db tech project led by computer scientists)
- Arkin lab's MicrobesOnline server has cool tree-based view of sequence families
- Hovergen, hogenom
- organism-centered gene-family databases, e.g., for plasmodium
Notes on how to approach this
Semantic transformation, ontologies, and formats
As Gopal said at one of our earlier meetings "There is a theory of interoperability, and it is semantic transformation". Interoperability is about expressing the same meaning in different ways, or about maintaining the meaning of information as it flows through different formats or dialects or contexts. This information may have a representation in the form of a string of characters, or a diagram, or some other kind of tokens. The meaning of tokens is formalized through ontologies and other standards.
If we want data resources, services, or software applications to interoperate in their treatment of a "coding sequence", we need to know what a "coding sequence" means for each one. If the gene_masher database assumes that "coding sequence" includes a stop codon, but the gene_whacker program assumes it doesn't, then we have a potential interoperability problem. Gene_whacker might crash if given a gene_masher coding sequence, or if gene_whacker is more carefully written software, it might refuse to validate a coding sequence from gene_masher because its length does not match the expected length.
Our notes from the working group meeting
Data resource interop project
- need to develop critical mass for nexml-cdao (critical mass of community involvement, use-cases, eyes on code)
- hard to get critical mass by targeting app developers
- hard to show value by targeting app developers
- strategy of targeting data resources puts data in hands of users
- however there also are carrots for db maintainers:
- interchange with other sources
- common treatment of keys such as taxonomy
- data resources to target (see Data Resources page)
- implementation strategy
- recruit participants
- possible leadership from Hilmar, other suggestions (Rod Page, Bill Piel, Encyc of Life, TOL)
- other participants drawn from projects above
- develop project plan
- consider implementing a coordinated db with families from many sources (similar philosophy to InterPro for protein family alignment dbs?)
- CDAO to providing constraining vocabulary for nexml schema
- disagreement over whether this is a good idea
- to succeed, we must provide users a way to expand nexml, thus to expand cdao
- mechanism is the feature request, e.g., I want to represent Bremer values
- user contacts nexml.org about how to represent new type of data
- if a formal mechanism is desired but does not exist
- user is referred to cdao feature request (must provide searchable interface and front page link to feature request form)
- cdao developers are obliged to respond
- get support
- NSF grant
- NESCent-sponsored hackathon
- recruit participants
More notes, from telecon with Hilmar and Arlin (5/29/08)
Outline of the basic plan
- expand CDAO to support more metadata
- map CDAO to a relational schema (CDAO already is mapped to nexml and NEXUS)
- develop a database interface that has some nice features
- better query interface than TreeBase or Pandit because it has more access to semantics
- output in rich, structured format (nexml)
- some nice integrated tools such as ATV and jalview, iTOL
- pick two or three data resource managers to come work with us to develop a mapping, so that the content of their resources can be uploaded automatically into the database application via nexml-CDAO
- finally, hold a hackathon where we bring in other data resource managers to create their own mappings
- the goal is to develop an integration platform
- integrate taxonomy, phylogeny, gene family databases
- data resource manager can take existing content, load it into db package, and automatically get the cool features
- taxonomic links
- standard services API
- allows data submission, returns data in nexml or equiv
- accepts metadata and transmits it
- reasons over metadata, e.g., validate MIAPA compliance
- NSF grant
- Advances in Biological Informatics (due August 12)
- program has explicit shift in focus away from database creation . . .
- . . . but that's ok because we won't focus on One Database That Rules Them All, but on the infrastructure that supports them all
- isn't this proposing the same thing as pPOD?
- apparent overlap with pPOD is strong; need buy-in from them
- do we need a pilot project to get preliminary results?
- do we need to hire interface programmers to do this? A: can do it by contract, at least initially