April 2004 - tdwg-content - lists.tdwg.org

Re: SEEK Project and TDWG-SDD
by Donald Hobern 16 Apr '04

16 Apr '04

After reading this thread, I'd like to return to the key issue raised by Jim in his initial post: > Bryan described to us how your schema is intended in include all of the > data and metadata associated with a diagnostic character set for a group > of taxa. My understanding is thus incomplete and based only on our > single discussion, but it struck me that TDWG-SDD has an opportunity to > have much broader acceptance and support if your schema was not designed > as a single data object--to contain both the metadata about the package > (or work or whatever you refer to it as) *and* the descriptive data that > describe the individual concepts. > > If the taxa/concepts had their own schemas and were linked to the > package metadata with a GUID, maybe a DOI or some other globally unique > identifier, then the XML concept data sets could be used for other > systems like concept based classification or database management > systems. This would in theory, and in my view, give the work of your > group much more leverage, exposure and relevance to a broader group of > scientists and users of names and concepts. I know that the SDD team have already been considering these issues (in conjunction with the ABCD team and others), but I would like to make the following points reflecting my priorities here in GBIF. 1. It will certainly be beneficial to model the data elements from SDD in such a way that they can be reused in other documents. I would think that this should ideally allow at least for the Terminology, Entities, Descriptions and Keys elements to form independent schema elements which could be used in other schemas. We do not want the use of these elements to be restricted just because they have been tightly bound to a specific document schema. The intention with the ABCD schema was in part to start the development of a library of reusable XML biodiversity data types (not just a fixed document structure). I see much of the current work of TDWG (and of GBIF) to be developing just such a library, from which we can seek to compose a wide variety of top level document structures (specimen/observation data, character tables, diagnostic keys, taxonomic revisions, markup of legacy taxonomic literature like the Biologia Centrali-Americana). 2. On the other hand, it does seem sensible to provide for the metadata to be transferred with each data set so that ownership information, usage restrictions, known limitations, etc. are not lost. I would therefore like to put effort into adopting or developing a top-level document envelope suitable for all classes of biodiversity data exchange. This should include information on origin and ownership, data transformation history, taxonomic, geographic and temporal coverage (as appropriate) and any metadata necessary to allow processors to identify the schema(s) in use for the actual data within the document. The real content should be separable for re-use in other contexts, but such a metadata wrapper standard would bring us closer to automating the manipulation of a wide range of content. I have in mind here something like the ABCD structure, with a top level DataSets wrapper containing a number of different DataSet objects, each of which is made up of a set of Units. In effect the Units element would be a container for data elements from SDD, ABCD, TDWG-Names, etc. The DataSet-level elements would provide a common metadata model for all of these documents. I feel that we need a grand vision for how we will unify all of these different schemas into an overall information model. Consider the task of developing a full taxonomic revision for a group using XML documents. This would naturally include references to specimens underlying a concept (external references to ABCD documents?), character data (SDD), and nomenclatural data (TDWG-Names). The goal should be for a processor to be able to take such a document and treat it as an element within a comprehensive electronic library of biodiversity data. We will need registries of the locations of different documents (with some form of GUID for each document) and mechanisms for managing the cross-references (taxon names, catalog numbers for specimens, author names, character definitions, etc.). Such an infrastructure would allow us to populate our taxon concepts with all of the relevant information. Donald --------------------------------------------------------------- Donald Hobern (dhobern(a)gbif.org) Programme Officer for Data Access and Database Interoperability Global Biodiversity Information Facility Secretariat Universitetsparken 15, DK-2100 Copenhagen, Denmark Tel: +45-35321483 Mobile: +45-28751483 Fax: +45-35321480 ---------------------------------------------------------------

1 0

Re: [SEEK-Taxon] SEEK Project and TDWG-SDD
by ram 15 Apr '04

15 Apr '04

Hoping to entice critics to the SDD Wiki, I argue at http://efgblade.cs.umb.edu/twiki/bin/view/SDD/ExternalTerminology that GUIDs are not alone sufficient to address important issues arising from separating character and state definitions from descriptions. If you try to respond on the Wiki, you'll be invited to register, which will be immediately accepted. The start of the SDD Wiki is at http://efgblade.cs.umb.edu/twiki/bin/view/SDD/WebHome Bob

1 0

Fwd: Re: SEEK Project and TDWG-SDD
by P. Bryan Heidorn 15 Apr '04

15 Apr '04

Date: Thu, 15 Apr 2004 20:03:07 -0500 To: TDWG - Structure of Descriptive Data <TDWG-SDD(a)LISTSERV.NHM.KU.EDU> From: "P. Bryan Heidorn" <pheidorn(a)uiuc.edu> Subject: Re: SEEK Project and TDWG-SDD Cc: pheidorn(a)uiuc.edu My notes below are pushing it a bit far but i had time to think about this on the plane back from Kansas and the SEEK project today. At 11:44 AM 4/15/2004, Jim Beach wrote: >On Thu, 15 Apr 2004 10:54:36 -0500, Julian H <humphries(a)MAIL.UTEXAS.EDU> >wrote: > > >At 10:44 AM 4/15/2004, you wrote: > >>single discussion, but it struck me that TDWG-SDD has an opportunity to > >>have much broader acceptance and support if your schema was not designed > >>as a single data object--to contain both the metadata about the package > >>(or work or whatever you refer to it as) *and* the descriptive data that > >>describe the individual concepts. > > > >Another novice (pre-novice) here, are you specifically referring to > >separating out taxonomic concept information (metadata) from the > >descriptive data? > >No, I was thinking of seperating the metadata about the package "This is a >data set of Magnolias from FNA, it was assembled by, organized by, dates, >etc.) from the data describing the character states of the individual >taxa. So a good question is what do you do with the character >definitions! It seems the character state values without the character >definitions would not be of much use for any system to interpret the >meaning of the states. Two options, de-normalize the character >definitions and put them in each concept schema, or two have a separate >server, and an external reference in the data schema that, has the >character definitions. Not sure how that choice would play out. To this point we avoided the character naming issue, I think in part because of the controversy that is associated with his issue. The general external reference mechanism described in Gregor's later post, is an attempt to push off that controversy, in part to a system outside of SDD. That does have the advantage of simplifying SDD a bit which is greatly needed. It might be worth considering how the character definition service might actually work. Building on Jim's ideas, we could imagine a mechanism that would allow anyone (with authority) to add definitions and globally unique identifiers for the characters. It may beyond our financial means now but there could be a service where one could send a character / or state definition, including a current context of application. It would be date stamped. Open to some very minimal revision permission. New global ideas would need to be created is the definition were changed or expanded. Of course the definition of a character might need to refer to a other registered character groups, characters or states. if GUCID is a global unique character ID You might sent the registration service the following XML document and get a GUCID back in return. Now any reference to this group in any species description in earth could cite this GUCID as part of a description. In this definition we could include GUCID references to the necessary components. <CharacterGroup> <CharacterGroupName name="flowers" GUCID=f???"> <Context>angiosperms</Context> flowers</CharacterGroupName> <LegalValue name="inflorescence_position" GUCID="1ej48dhk"></LegalValue> <LegalValue name="inflorescence_type" GUCID=NSNJKNDJBY248N"></LegalValue> Many more here <Definition>Sexual reproduction apparatus of a plant</Definition> <Synonym></Synonym> <BroaderTerm></BroaderTerm> <NarrowerTerm></NarrowerTerm> <RelatedTerm></RelatedTerm> </CharacterGroup> using a collection of character definitions defined outside you could have a stand alone description. Given two taxonomic descriptions, you could decide if they are using the same definitions for their characters by looking at the GUCIDs. I well formed character matrix could be constructed from the intersection of like characters in a collection of taxonomic descriptions. I well organized project might want to decide which characters to use ahead of the building the descriptions to make sure there are not too many conflicts in the definitions. Perhaps context would help with some of the conflicts. Not that two projects could use the word "flower" to mean two completely different things, and a computer programs could know this because they would have different GUCIDs There is no reason not to create character types and registration to handle gene sequences or whatever. I think this is already covered very well in SDD. In fact, I think the character definition section of SDD could be used almost exactly the way it is now except that we would rely on the existence of the global character registry. I know this is a bit radical and maybe heavy handed but i do not think it is all that difficult given the structures already in SDD. I think we would still need the certainty mechanisms in SDD to support the certainty or prevalence of a character/state within an individual species or taxon. (almost always red flowers.) > > > >>If the taxa/concepts had their own schemas and were linked to the > >>package metadata with a GUID, maybe a DOI or some other globally unique > >>identifier, then the XML concept data sets could be used for other > >>systems like concept based classification or database management > >>systems. >....snip..... >if the taxon data sets (and maybe also their character definitions)we in >sperate XML documents, then we could use them as fodder for other concept >systems. > > >... snip ..... > >incomplete SDD data sets? More on dataset archives in the next email. > >People serving SDD data sets thorugh the web, would presumably be aware of >data set integrity issues and make sure their SDD packages were complete. I think the registration of the individual taxonomic descriptions is another very parallel issue. Relatively easy to do either as collections of treatments as are now in SDD or as standalone treatments with globally defined character sets. Regards, Bryan -------------------------------------------------------------------- P. Bryan Heidorn Graduate School of Library and Information Science pheidorn(a)uiuc.edu University of Illinois at Urbana-Champaign MC-493 (V)217/ 244-7792 Rm. 221, 501 East Daniel St., Champaign, IL 61820-6212 (F)217/ 244-3302 http://alexia.lis.uiuc.edu/~heidorn Calendar: http://calendar.yahoo.com/pbheidorn Visit the Biobrowser Web site at http://www.biobrowser.org, http://www.isrl.uiuc.edu/~telenature, http://www.isrl.uiuc.edu/~openkey

1 0

Re: SEEK Project and TDWG-SDD
by Gregor Hagedorn 15 Apr '04

15 Apr '04

> No, I was thinking of seperating the metadata about the package "This > is a data set of Magnolias from FNA, it was assembled by, organized > by, dates, etc.) from the data describing the character states of the > individual taxa. So a good question is what do you do with the > character definitions! It seems the character state values without the > character definitions would not be of much use for any system to > interpret the meaning of the states. Two options, de-normalize the > character definitions and put them in each concept schema, or two have > a separate server, and an external reference in the data schema that, > has the character definitions. Not sure how that choice would play > out. This is on our agenda, but I am stuggling how to do it. What we already know is: SDD can have a project that only defines terminology. We anticipate that other projects containing descriptions would like to use a common terminology project. Perhaps even better: select blocks from multiple such projects. The problem is how to organize this. One problem is technical: the xml schema identity constraints only work within a single file, which I think is a stupid design bug in xml schema. The second problem is that the obvious solution would be to use some kind of xinclude that inserts one file into another. We would need inserting fragments, or split the terminology exactly at some "block level" (modules of several characters or concepts) into files. My experience is that xml tools do not support xinclude. I mostly work with xml spy, and I could not get it to work. Does anybody have experience how this modular design can be solved with current tools, so that we can do some testing? Which examples exist that have a similar modular structure, but still use identifiers the referential integrity of which should be validated? > >>If the taxa/concepts had their own schemas and were linked to the > >>package metadata with a GUID, maybe a DOI or some other globally > >>unique identifier, then the XML concept data sets could be used for > >>other systems like concept based classification or database > >>management systems. What is a DOI? Wouldn't I need more than the GUID information? Currently we achieve GUIDs through combining a Project GUID with local 32 bit integer IDs. I personally am a fan of the long GUIDs for such purposes, but the majority prefers URI-type IDs. The Project GUID is planned either to be a self-controlled URL, or a registered Project name, e.g. registered through a GBIF UDDI. > > >The overhead for the traditional diagnostic identification software > >>makers would be that the XML parts would need to assembled for the > >>various applications that use the data and there would be the > >>potential risk that SDD data sets would be incomplete, if there were > >>some careless file management. > > > >What parts could get lost? the taxonomic parts? > > yes, if you had multiple xml docs for the same 'diagnostic package' > they would be managed as distinct files. Already asked above: should it really be files? Can we not have fragment identifiers to point within files? What are the techniques to do this? Gregor ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: SEEK Project and TDWG-SDD
by Gregor Hagedorn 15 Apr '04

15 Apr '04

Hi Jim! > Well, I hesitate to stick my neck out and into the space of this > august, expert group Actually this expert group is small and struggling with a complex design. Too few people have the time either to penetrate it and find the critical points still lacking, or to write comprehensive documentation. Any contributions are most welcome! > (http://seek.ecoinformatics.org), with which I see there has been some > interaction with SDD, is attempting to build a global infrastructure > for handling, resolving and communicating taxonomic concepts (as > opposed to taxonomic names). .... and as I see it, the most valuable tax. concepts are not based on synonymy or specimen lists, but on comprehensive descriptions. .... which makes me critical of the current lack of providing name identifiers separately from taxon concept identifiers. To describe a new concept in SDD we need name, not concept IDs... But this is a GBIF ECAT issue... > If the taxa/concepts had their own schemas and were linked to the > package metadata with a GUID, maybe a DOI or some other globally > unique identifier, then the XML concept data sets could be used for > other systems like concept based classification or database management > systems. This would in theory, and in my view, give the work of your > group much more leverage, exposure and relevance to a broader group of > scientists and users of names and concepts. This is right into the heart of our discussions. We absolutely do NOT want to define names, we want to use them. You may say link to them. The current schema contains a general attempt to link to external resources, including specimens and taxonomic names. See Entities/Objects and Entities/Classes in the schema. We call these objects ResourceConnectors. Each connector has an external ID, a provider, and a human-readable text representation. In addition, the derived connector types may add additional information. The name "ResourceConnector" may be vague or confusing. In a way, it is intended as a proxy design pattern, each connector stands as a proxy object for a external object, the link to which may be temporarily unavailable. Any suggestions what better name to call this? > some careless file management. But presumably you guys are thinking > about a registry or distributed federation of these data sets anyway, > where they would be archived and served intact from a trusted source. Rather struggling with UDDI... I am still confused what, besides a UDDI URL should be stored to reproducible retrieve a service wsdl URI (that may change over time, thus the use of a UDDI). UDDI is defined extremely general, too general for me. Also, we would need to develop interface definitions containing a set of standard operations to retrieve lists of external objects to make a selection from, or to retrieve rich information about an external object (i.e. more than the proxy inside SDD saves. I believe this is essential for the future of GBIF but wonder where the resources are to tackle these problems... > I also understand that data sets of diagnostic identification > information are far from complete descriptions of concepts in either a > taxonomic or phylogenetic sense, but if the SDD concept schema could > accommodate additional characters, then the opportunity would be there > for other people to use SDD for other kinds of systems. The UI of > diagnostic key programs would likely not need to use or display DNA > sequences for interactive identification, but no harm done, they could > just ignore fields of no use to the program at hand. I believe this is a misunderstanding. SDD is in no ways confined to diagnostic sets, if diagnostic is understood as a minimized set of field-observable characters. That may work for plants, it does not work for fungi or other microorganisms. We do attempt to provide means to differentiate between different kind of characters. One method is that concept trees point to the characters, and the method tree contains different methods. Another is an attempt to allow ratings to rate characters and concepts (the characters would inherit from concepts) for identification convenience, identification availability, identification reliability of feature DELTA had a simple character wheight. Problem with that is that it had no semantics other than change it until your selected program produces a desired result. Any comments or experience with this? > Another objection I anticipate would be that broadening SDD to > accommodate the functional requirements of the broader taxon concept > management objectives would make the SDD schema too complex and > difficult for anyone to work with. No, I think that is where SDD has to go. I guess difficult is where we are already... We try to break it up into modular components to show that SDD tackles a range of problems, and that smaller subsets of SDD are available for individual problems. Can you provide some ideas about how you would modularize the data, with just a keyword /example list of things in each module. I think without you looking in detail at the schema, such a list might be very valuable to the development of SDD! Thanks a lot! Gregor ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: SEEK Project and TDWG-SDD
by Robert A. Morris 15 Apr '04

15 Apr '04

Jim Beach wrote: > On Thu, 15 Apr 2004 10:54:36 -0500, Julian H <humphries(a)MAIL.UTEXAS.EDU> > wrote: > > >>At 10:44 AM 4/15/2004, you wrote: >> >>>single discussion, but it struck me that TDWG-SDD has an opportunity to >>>have much broader acceptance and support if your schema was not designed >>>as a single data object--to contain both the metadata about the package >>>(or work or whatever you refer to it as) *and* the descriptive data that >>>describe the individual concepts. >> >>Another novice (pre-novice) here, are you specifically referring to >>separating out taxonomic concept information (metadata) from the >>descriptive data? > > > No, I was thinking of seperating the metadata about the package "This is a > data set of Magnolias from FNA, it was assembled by, organized by, dates, > etc.) from the data describing the character states of the individual > taxa. So a good question is what do you do with the character > definitions! It seems the character state values without the character > definitions would not be of much use for any system to interpret the > meaning of the states. Two options, de-normalize the character > definitions and put them in each concept schema, or two have a separate > server, and an external reference in the data schema that, has the > character definitions. Not sure how that choice would play out. > First,it is important to understand that there is NO schema for any particular taxon, group of taxa, specimen, or any other thing or class of things that an SDD document can describe--such as, diseases, restaurants of Lawrence Kansas, "Halcyon House Bed and Breakfast", or avian pests of Boston, MA. Rather the SDD schema constrains how you make descriptions, how you represent properties and values (i.e. characters and states), howthose things can be related to one another, and how you can make decision trees based on them, usually for identification purposes. Bearing in mind that SDD is in its rev 0.9, which is a proposal not yet adopted by TDWG, in its "Terminology" section, SDD provides for both "global" and "local" states, the former applicable to any characters that wishes to use them. Examples might be various colors, shape terms like "ovate", etc. Consuming applications are invited to use a semantics in which global states are identical no matter where used. (For local states this is not a meaningful question in SDD. ). Characters and states in an SDD document do not get a GUID (though it sounds worth considering), so documents that do not use the same Terminology section can't compare characters or states in any defined way. Also the only mechanism on the table for sharing Terminology across instance documents is by XML inclusion, and this is not really a persistant mechanism suitable for integration. In the present version, it is generally expected that all the Terminology is defined if not in the document, then in something that accompanies it, thus finessing the issue. Put another way, SDD's first design target was data exchange, not data integration, with informal attempt to keep in view where the issues may be to migrate to support of integrating applications. Put yet another way, to reliably use an SDD0.9 instance refering to shared external Terminology requires a contract between all such instances that the external Terminology is the same. SDD0.9 itself provides no mechanisms for representing or enforcing such a contract, nor a mechanism for expressing a fallback position if an application can represent and detect such a violation. (Though at most expected places, SDD provides for application-specific data to be inserted of which applications can make whatever meaning they wish). I don't doubt that it is easy to replace inclusion with a reference to a shared Terminolgy acquired through a registry and accessed by a GUID or DOI. I'm certain that the only issue would be what to do if the Terminology is in fact unacquirable, and that issue is orthogonal to all the structure issues that SDD is meant to address. The SDD committee welcomes both lurkers and contributers to the SDD discussion Wiki http://efgblade.cs.umb.edu/twiki/bin/view/SDD/WebHome There you can especially see what questions we wrestled with, and can especially contribute questions we missed or comment on answers that are less than helpful in other contexts where we missed a chance to contribute. We expect there will be an SDD workshop before TDWG in Christ Church and hope that well before then there will be several implementations to study. > > >>>If the taxa/concepts had their own schemas and were linked to the >>>package metadata with a GUID, maybe a DOI or some other globally unique >>>identifier, then the XML concept data sets could be used for other >>>systems like concept based classification or database management >>>systems. >> >>Could you write this sentence with a few more words? I'm want to be sure > > I > >>get the concept. > > > How about an ASCII graphic? I'm on thin ice, but if the metadata for the > package is this: > > MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM > > and the individual taxon character state sets are this: > > taxon1 taxon2 taxon3 > char-val-1 char-val-1 etc. > char-val-2 etc. > char-val-3 > > if the taxon data sets (and maybe also their character definitions)we in > sperate XML documents, then we could use them as fodder for other concept > systems. > > > >>The overhead for the traditional diagnostic identification software >> >>>makers would be that the XML parts would need to assembled for the >>>various applications that use the data and there would be the potential >>>risk that SDD data sets would be incomplete, if there were some careless >>>file management. >> >>What parts could get lost? the taxonomic parts? > > > yes, if you had multiple xml docs for the same 'diagnostic package' they > would be managed as distinct files. > > >> >>> But presumably you guys are thinking about a registry >>>or distributed federation of these data sets anyway, where they would be >>>archived and served intact from a trusted source. >> >>Um, now I am really lost, amplify please? What does this have to do with >>incomplete SDD data sets? More on dataset archives in the next email. > > > People serving SDD data sets thorugh the web, would presumably be aware of > data set integrity issues and make sure their SDD packages were complete. > > Yes, as above, that is the present assumption of SDD. >> >>>I also understand that data sets of diagnostic identification >>>information are far from complete descriptions of concepts in either a >>>taxonomic or phylogenetic sense, but if the SDD concept schema could >>>accommodate additional characters, then the opportunity would be there >>>for other people to use SDD for other kinds of systems. The UI of >>>diagnostic key programs would likely not need to use or display DNA >>>sequences for interactive identification, but no harm done, they could >>>just ignore fields of no use to the program at hand. >> >> >>Ok, now we are getting to something I know about. See the next email for >>some comments on this... >> >>Julian >> >> >> >>Julian Humphries >>DigiMorph.Org >>Geological Sciences >>University of Texas at Austin >>Austin, TX 78712 >>512-471-3275 -- Robert A. Morris, Professor of Computer Science University of Massachusetts at Boston 100 Morrissey Blvd; Boston, MA 02125 http://www.cs.umb.edu/~ram http://www.cs.umb.edu/efg phone: (+1)617-287-6466 fax: (+1)617-287-6433

1 0

Re: SEEK Project and TDWG-SDD
by Julian H 15 Apr '04

15 Apr '04

At 12:57 PM 4/15/2004, you wrote: > > No, I was thinking of seperating the metadata about the package "This > > is a data set of Magnolias from FNA, it was assembled by, organized > > by, dates, etc.) from the data describing the character states of the >... >The problem is how to organize this. One problem is technical: the >xml schema identity constraints only work within a single file, which >I think is a stupid design bug in xml schema. > >The second problem is that the obvious solution would be to use some >kind of xinclude that inserts one file into another. We would need >inserting fragments, or split the terminology exactly at some "block >level" (modules of several characters or concepts) into files. Xlink? Won't most of this happen at some kind of web service level? Application builders will need to know what schemas to draw on for different components of their tools, but the complexity of the underlying documents shouldn't be too much of a barrier. On the other hand, maintaining referential integrity (and resolving conflicting standards for similar concepts) across multiple standards may be a real challenge. Fundamentally, Jim's question about where to define "characters" addresses this in a real world way. For truly simple, widely accepted character concepts either solution works, but where definitions diverge, both solution have problems. I think this has to be denormalized and rely on data mining tools to rationalize equivalence of concepts across multiple definitions of "flower color." BTW, there is an interesting article on bioinformatic (my spell checker suggests 'bondwoman, bookworm and bonbons' for bioinformatic!) databases (focusing on PDB) in this weeks Nature. They at least mention some of these issues. Julian Julian Humphries DigiMorph.Org Geological Sciences University of Texas at Austin Austin, TX 78712 512-471-3275

1 0

Re: SEEK Project and TDWG-SDD
by Jim Beach 15 Apr '04

15 Apr '04

On Thu, 15 Apr 2004 10:54:36 -0500, Julian H <humphries(a)MAIL.UTEXAS.EDU> wrote: >At 10:44 AM 4/15/2004, you wrote: >>single discussion, but it struck me that TDWG-SDD has an opportunity to >>have much broader acceptance and support if your schema was not designed >>as a single data object--to contain both the metadata about the package >>(or work or whatever you refer to it as) *and* the descriptive data that >>describe the individual concepts. > >Another novice (pre-novice) here, are you specifically referring to >separating out taxonomic concept information (metadata) from the >descriptive data? No, I was thinking of seperating the metadata about the package "This is a data set of Magnolias from FNA, it was assembled by, organized by, dates, etc.) from the data describing the character states of the individual taxa. So a good question is what do you do with the character definitions! It seems the character state values without the character definitions would not be of much use for any system to interpret the meaning of the states. Two options, de-normalize the character definitions and put them in each concept schema, or two have a separate server, and an external reference in the data schema that, has the character definitions. Not sure how that choice would play out. > >>If the taxa/concepts had their own schemas and were linked to the >>package metadata with a GUID, maybe a DOI or some other globally unique >>identifier, then the XML concept data sets could be used for other >>systems like concept based classification or database management >>systems. > >Could you write this sentence with a few more words? I'm want to be sure I >get the concept. How about an ASCII graphic? I'm on thin ice, but if the metadata for the package is this: MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM and the individual taxon character state sets are this: taxon1 taxon2 taxon3 char-val-1 char-val-1 etc. char-val-2 etc. char-val-3 if the taxon data sets (and maybe also their character definitions)we in sperate XML documents, then we could use them as fodder for other concept systems. >The overhead for the traditional diagnostic identification software >>makers would be that the XML parts would need to assembled for the >>various applications that use the data and there would be the potential >>risk that SDD data sets would be incomplete, if there were some careless >>file management. > >What parts could get lost? the taxonomic parts? yes, if you had multiple xml docs for the same 'diagnostic package' they would be managed as distinct files. > > >> But presumably you guys are thinking about a registry >>or distributed federation of these data sets anyway, where they would be >>archived and served intact from a trusted source. > >Um, now I am really lost, amplify please? What does this have to do with >incomplete SDD data sets? More on dataset archives in the next email. People serving SDD data sets thorugh the web, would presumably be aware of data set integrity issues and make sure their SDD packages were complete. > > >>I also understand that data sets of diagnostic identification >>information are far from complete descriptions of concepts in either a >>taxonomic or phylogenetic sense, but if the SDD concept schema could >>accommodate additional characters, then the opportunity would be there >>for other people to use SDD for other kinds of systems. The UI of >>diagnostic key programs would likely not need to use or display DNA >>sequences for interactive identification, but no harm done, they could >>just ignore fields of no use to the program at hand. > > >Ok, now we are getting to something I know about. See the next email for >some comments on this... > >Julian > > > >Julian Humphries >DigiMorph.Org >Geological Sciences >University of Texas at Austin >Austin, TX 78712 >512-471-3275

1 0

Re: TDWG-SDD and morphological data sets
by Julian H 15 Apr '04

15 Apr '04

At 10:44 AM 4/15/2004, you wrote: >I also understand that data sets of diagnostic identification >information are far from complete descriptions of concepts in either a >taxonomic or phylogenetic sense, but if the SDD concept schema could >accommodate additional characters, then the opportunity would be there >for other people to use SDD for other kinds of systems. The UI of >diagnostic key programs would likely not need to use or display DNA >sequences for interactive identification, but no harm done, they could >just ignore fields of no use to the program at hand. Jim questions/comments encouraged me to bring up a somewhat related question about SDD. I am involved in a project to provide a variety of tools for collaboratively creating, managing and annotating digital morphological (DM) data. By and large, DM means pixel and voxel (mostly voxel) data derived from various tomographic instrumentation, but the system will also allow collection and storage of scanned illustrations or even just words and numbers. My initial inclination for such a system (at its most basic, a taxon by character matrix) was that it primarily served as a tool for collaborative building of matrices for phylogenetic analysis (but allowing all to see the actual character states rather than simply a terse word description), but quickly it became obvious that some folks might store regular descriptive data. Thus, I started thinking about how this would communicate with a variety of outside tools: tree building (Nexus), keys (Delta, SDD), etc. The system will also effectively act as an archive for published data sets (not in the formal sense at this point, but simply because the data reside in a network accessible database) with the added advantage of users being able to actually see and interact with the data that produced the conclusions seen in a paper. The point of this long introduction is that I want to reinforce the idea that SDD concept schema should be broad enough to accommodate additional character types, including not only sequence/protein/gene data but digital proxies of a variety of morphological data types as well. The "dentary" in a voxel data set might be represented by a database structure that encodes which voxels in a tomographic data brick belong to that particularly structure. If I need to pass this information on to an interactive identification tool, will SDD have a schema for passing the information? Looking for magic bullets.... Julian Julian Humphries DigiMorph.Org Geological Sciences University of Texas at Austin Austin, TX 78712 512-471-3275

1 0

[SEEK-Taxon] SEEK Project and TDWG-SDD
by Beach, James H 15 Apr '04

15 Apr '04

TDWG-SDD folk: One more thought which I should have mentioned, the SEEK project is working with GBIF through Jesse Kennedy's Napier University lab on a SEEK concept schema with the objective supporting concept-based databases, IR of concepts and other fun stuff such as web-based services for concept identification, resolution and disambiguation. I believe Jessie plans to present that schema at the TDWG meeting this year. I guess the logical end point of my argument would be that the two standards might become one, so that taxonomic concepts could be used by both groups of scientists from a single schema. It seems inevitable that GBIF, SEEK, Berlin, or whomever will put together a satisfactory, implementable, concept schema within the next year or two. My only thought is that the SDD standard might, with what seems to be a pretty minor shift in organization, be part of the broader concept effort. Excuse, if you all have been through this discussion N times! Jim B.

1 0