- tdwg-content - lists.tdwg.org

Re: SEEK Project and TDWG-SDD
by Gregor Hagedorn 15 Apr '04

15 Apr '04

Hi Jim! > Well, I hesitate to stick my neck out and into the space of this > august, expert group Actually this expert group is small and struggling with a complex design. Too few people have the time either to penetrate it and find the critical points still lacking, or to write comprehensive documentation. Any contributions are most welcome! > (http://seek.ecoinformatics.org), with which I see there has been some > interaction with SDD, is attempting to build a global infrastructure > for handling, resolving and communicating taxonomic concepts (as > opposed to taxonomic names). .... and as I see it, the most valuable tax. concepts are not based on synonymy or specimen lists, but on comprehensive descriptions. .... which makes me critical of the current lack of providing name identifiers separately from taxon concept identifiers. To describe a new concept in SDD we need name, not concept IDs... But this is a GBIF ECAT issue... > If the taxa/concepts had their own schemas and were linked to the > package metadata with a GUID, maybe a DOI or some other globally > unique identifier, then the XML concept data sets could be used for > other systems like concept based classification or database management > systems. This would in theory, and in my view, give the work of your > group much more leverage, exposure and relevance to a broader group of > scientists and users of names and concepts. This is right into the heart of our discussions. We absolutely do NOT want to define names, we want to use them. You may say link to them. The current schema contains a general attempt to link to external resources, including specimens and taxonomic names. See Entities/Objects and Entities/Classes in the schema. We call these objects ResourceConnectors. Each connector has an external ID, a provider, and a human-readable text representation. In addition, the derived connector types may add additional information. The name "ResourceConnector" may be vague or confusing. In a way, it is intended as a proxy design pattern, each connector stands as a proxy object for a external object, the link to which may be temporarily unavailable. Any suggestions what better name to call this? > some careless file management. But presumably you guys are thinking > about a registry or distributed federation of these data sets anyway, > where they would be archived and served intact from a trusted source. Rather struggling with UDDI... I am still confused what, besides a UDDI URL should be stored to reproducible retrieve a service wsdl URI (that may change over time, thus the use of a UDDI). UDDI is defined extremely general, too general for me. Also, we would need to develop interface definitions containing a set of standard operations to retrieve lists of external objects to make a selection from, or to retrieve rich information about an external object (i.e. more than the proxy inside SDD saves. I believe this is essential for the future of GBIF but wonder where the resources are to tackle these problems... > I also understand that data sets of diagnostic identification > information are far from complete descriptions of concepts in either a > taxonomic or phylogenetic sense, but if the SDD concept schema could > accommodate additional characters, then the opportunity would be there > for other people to use SDD for other kinds of systems. The UI of > diagnostic key programs would likely not need to use or display DNA > sequences for interactive identification, but no harm done, they could > just ignore fields of no use to the program at hand. I believe this is a misunderstanding. SDD is in no ways confined to diagnostic sets, if diagnostic is understood as a minimized set of field-observable characters. That may work for plants, it does not work for fungi or other microorganisms. We do attempt to provide means to differentiate between different kind of characters. One method is that concept trees point to the characters, and the method tree contains different methods. Another is an attempt to allow ratings to rate characters and concepts (the characters would inherit from concepts) for identification convenience, identification availability, identification reliability of feature DELTA had a simple character wheight. Problem with that is that it had no semantics other than change it until your selected program produces a desired result. Any comments or experience with this? > Another objection I anticipate would be that broadening SDD to > accommodate the functional requirements of the broader taxon concept > management objectives would make the SDD schema too complex and > difficult for anyone to work with. No, I think that is where SDD has to go. I guess difficult is where we are already... We try to break it up into modular components to show that SDD tackles a range of problems, and that smaller subsets of SDD are available for individual problems. Can you provide some ideas about how you would modularize the data, with just a keyword /example list of things in each module. I think without you looking in detail at the schema, such a list might be very valuable to the development of SDD! Thanks a lot! Gregor ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: SEEK Project and TDWG-SDD
by Robert A. Morris 15 Apr '04

15 Apr '04

Jim Beach wrote: > On Thu, 15 Apr 2004 10:54:36 -0500, Julian H <humphries(a)MAIL.UTEXAS.EDU> > wrote: > > >>At 10:44 AM 4/15/2004, you wrote: >> >>>single discussion, but it struck me that TDWG-SDD has an opportunity to >>>have much broader acceptance and support if your schema was not designed >>>as a single data object--to contain both the metadata about the package >>>(or work or whatever you refer to it as) *and* the descriptive data that >>>describe the individual concepts. >> >>Another novice (pre-novice) here, are you specifically referring to >>separating out taxonomic concept information (metadata) from the >>descriptive data? > > > No, I was thinking of seperating the metadata about the package "This is a > data set of Magnolias from FNA, it was assembled by, organized by, dates, > etc.) from the data describing the character states of the individual > taxa. So a good question is what do you do with the character > definitions! It seems the character state values without the character > definitions would not be of much use for any system to interpret the > meaning of the states. Two options, de-normalize the character > definitions and put them in each concept schema, or two have a separate > server, and an external reference in the data schema that, has the > character definitions. Not sure how that choice would play out. > First,it is important to understand that there is NO schema for any particular taxon, group of taxa, specimen, or any other thing or class of things that an SDD document can describe--such as, diseases, restaurants of Lawrence Kansas, "Halcyon House Bed and Breakfast", or avian pests of Boston, MA. Rather the SDD schema constrains how you make descriptions, how you represent properties and values (i.e. characters and states), howthose things can be related to one another, and how you can make decision trees based on them, usually for identification purposes. Bearing in mind that SDD is in its rev 0.9, which is a proposal not yet adopted by TDWG, in its "Terminology" section, SDD provides for both "global" and "local" states, the former applicable to any characters that wishes to use them. Examples might be various colors, shape terms like "ovate", etc. Consuming applications are invited to use a semantics in which global states are identical no matter where used. (For local states this is not a meaningful question in SDD. ). Characters and states in an SDD document do not get a GUID (though it sounds worth considering), so documents that do not use the same Terminology section can't compare characters or states in any defined way. Also the only mechanism on the table for sharing Terminology across instance documents is by XML inclusion, and this is not really a persistant mechanism suitable for integration. In the present version, it is generally expected that all the Terminology is defined if not in the document, then in something that accompanies it, thus finessing the issue. Put another way, SDD's first design target was data exchange, not data integration, with informal attempt to keep in view where the issues may be to migrate to support of integrating applications. Put yet another way, to reliably use an SDD0.9 instance refering to shared external Terminology requires a contract between all such instances that the external Terminology is the same. SDD0.9 itself provides no mechanisms for representing or enforcing such a contract, nor a mechanism for expressing a fallback position if an application can represent and detect such a violation. (Though at most expected places, SDD provides for application-specific data to be inserted of which applications can make whatever meaning they wish). I don't doubt that it is easy to replace inclusion with a reference to a shared Terminolgy acquired through a registry and accessed by a GUID or DOI. I'm certain that the only issue would be what to do if the Terminology is in fact unacquirable, and that issue is orthogonal to all the structure issues that SDD is meant to address. The SDD committee welcomes both lurkers and contributers to the SDD discussion Wiki http://efgblade.cs.umb.edu/twiki/bin/view/SDD/WebHome There you can especially see what questions we wrestled with, and can especially contribute questions we missed or comment on answers that are less than helpful in other contexts where we missed a chance to contribute. We expect there will be an SDD workshop before TDWG in Christ Church and hope that well before then there will be several implementations to study. > > >>>If the taxa/concepts had their own schemas and were linked to the >>>package metadata with a GUID, maybe a DOI or some other globally unique >>>identifier, then the XML concept data sets could be used for other >>>systems like concept based classification or database management >>>systems. >> >>Could you write this sentence with a few more words? I'm want to be sure > > I > >>get the concept. > > > How about an ASCII graphic? I'm on thin ice, but if the metadata for the > package is this: > > MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM > > and the individual taxon character state sets are this: > > taxon1 taxon2 taxon3 > char-val-1 char-val-1 etc. > char-val-2 etc. > char-val-3 > > if the taxon data sets (and maybe also their character definitions)we in > sperate XML documents, then we could use them as fodder for other concept > systems. > > > >>The overhead for the traditional diagnostic identification software >> >>>makers would be that the XML parts would need to assembled for the >>>various applications that use the data and there would be the potential >>>risk that SDD data sets would be incomplete, if there were some careless >>>file management. >> >>What parts could get lost? the taxonomic parts? > > > yes, if you had multiple xml docs for the same 'diagnostic package' they > would be managed as distinct files. > > >> >>> But presumably you guys are thinking about a registry >>>or distributed federation of these data sets anyway, where they would be >>>archived and served intact from a trusted source. >> >>Um, now I am really lost, amplify please? What does this have to do with >>incomplete SDD data sets? More on dataset archives in the next email. > > > People serving SDD data sets thorugh the web, would presumably be aware of > data set integrity issues and make sure their SDD packages were complete. > > Yes, as above, that is the present assumption of SDD. >> >>>I also understand that data sets of diagnostic identification >>>information are far from complete descriptions of concepts in either a >>>taxonomic or phylogenetic sense, but if the SDD concept schema could >>>accommodate additional characters, then the opportunity would be there >>>for other people to use SDD for other kinds of systems. The UI of >>>diagnostic key programs would likely not need to use or display DNA >>>sequences for interactive identification, but no harm done, they could >>>just ignore fields of no use to the program at hand. >> >> >>Ok, now we are getting to something I know about. See the next email for >>some comments on this... >> >>Julian >> >> >> >>Julian Humphries >>DigiMorph.Org >>Geological Sciences >>University of Texas at Austin >>Austin, TX 78712 >>512-471-3275 -- Robert A. Morris, Professor of Computer Science University of Massachusetts at Boston 100 Morrissey Blvd; Boston, MA 02125 http://www.cs.umb.edu/~ram http://www.cs.umb.edu/efg phone: (+1)617-287-6466 fax: (+1)617-287-6433

1 0

Re: SEEK Project and TDWG-SDD
by Julian H 15 Apr '04

15 Apr '04

At 12:57 PM 4/15/2004, you wrote: > > No, I was thinking of seperating the metadata about the package "This > > is a data set of Magnolias from FNA, it was assembled by, organized > > by, dates, etc.) from the data describing the character states of the >... >The problem is how to organize this. One problem is technical: the >xml schema identity constraints only work within a single file, which >I think is a stupid design bug in xml schema. > >The second problem is that the obvious solution would be to use some >kind of xinclude that inserts one file into another. We would need >inserting fragments, or split the terminology exactly at some "block >level" (modules of several characters or concepts) into files. Xlink? Won't most of this happen at some kind of web service level? Application builders will need to know what schemas to draw on for different components of their tools, but the complexity of the underlying documents shouldn't be too much of a barrier. On the other hand, maintaining referential integrity (and resolving conflicting standards for similar concepts) across multiple standards may be a real challenge. Fundamentally, Jim's question about where to define "characters" addresses this in a real world way. For truly simple, widely accepted character concepts either solution works, but where definitions diverge, both solution have problems. I think this has to be denormalized and rely on data mining tools to rationalize equivalence of concepts across multiple definitions of "flower color." BTW, there is an interesting article on bioinformatic (my spell checker suggests 'bondwoman, bookworm and bonbons' for bioinformatic!) databases (focusing on PDB) in this weeks Nature. They at least mention some of these issues. Julian Julian Humphries DigiMorph.Org Geological Sciences University of Texas at Austin Austin, TX 78712 512-471-3275

1 0

Re: SEEK Project and TDWG-SDD
by Jim Beach 15 Apr '04

15 Apr '04

On Thu, 15 Apr 2004 10:54:36 -0500, Julian H <humphries(a)MAIL.UTEXAS.EDU> wrote: >At 10:44 AM 4/15/2004, you wrote: >>single discussion, but it struck me that TDWG-SDD has an opportunity to >>have much broader acceptance and support if your schema was not designed >>as a single data object--to contain both the metadata about the package >>(or work or whatever you refer to it as) *and* the descriptive data that >>describe the individual concepts. > >Another novice (pre-novice) here, are you specifically referring to >separating out taxonomic concept information (metadata) from the >descriptive data? No, I was thinking of seperating the metadata about the package "This is a data set of Magnolias from FNA, it was assembled by, organized by, dates, etc.) from the data describing the character states of the individual taxa. So a good question is what do you do with the character definitions! It seems the character state values without the character definitions would not be of much use for any system to interpret the meaning of the states. Two options, de-normalize the character definitions and put them in each concept schema, or two have a separate server, and an external reference in the data schema that, has the character definitions. Not sure how that choice would play out. > >>If the taxa/concepts had their own schemas and were linked to the >>package metadata with a GUID, maybe a DOI or some other globally unique >>identifier, then the XML concept data sets could be used for other >>systems like concept based classification or database management >>systems. > >Could you write this sentence with a few more words? I'm want to be sure I >get the concept. How about an ASCII graphic? I'm on thin ice, but if the metadata for the package is this: MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM and the individual taxon character state sets are this: taxon1 taxon2 taxon3 char-val-1 char-val-1 etc. char-val-2 etc. char-val-3 if the taxon data sets (and maybe also their character definitions)we in sperate XML documents, then we could use them as fodder for other concept systems. >The overhead for the traditional diagnostic identification software >>makers would be that the XML parts would need to assembled for the >>various applications that use the data and there would be the potential >>risk that SDD data sets would be incomplete, if there were some careless >>file management. > >What parts could get lost? the taxonomic parts? yes, if you had multiple xml docs for the same 'diagnostic package' they would be managed as distinct files. > > >> But presumably you guys are thinking about a registry >>or distributed federation of these data sets anyway, where they would be >>archived and served intact from a trusted source. > >Um, now I am really lost, amplify please? What does this have to do with >incomplete SDD data sets? More on dataset archives in the next email. People serving SDD data sets thorugh the web, would presumably be aware of data set integrity issues and make sure their SDD packages were complete. > > >>I also understand that data sets of diagnostic identification >>information are far from complete descriptions of concepts in either a >>taxonomic or phylogenetic sense, but if the SDD concept schema could >>accommodate additional characters, then the opportunity would be there >>for other people to use SDD for other kinds of systems. The UI of >>diagnostic key programs would likely not need to use or display DNA >>sequences for interactive identification, but no harm done, they could >>just ignore fields of no use to the program at hand. > > >Ok, now we are getting to something I know about. See the next email for >some comments on this... > >Julian > > > >Julian Humphries >DigiMorph.Org >Geological Sciences >University of Texas at Austin >Austin, TX 78712 >512-471-3275

1 0

Re: TDWG-SDD and morphological data sets
by Julian H 15 Apr '04

15 Apr '04

At 10:44 AM 4/15/2004, you wrote: >I also understand that data sets of diagnostic identification >information are far from complete descriptions of concepts in either a >taxonomic or phylogenetic sense, but if the SDD concept schema could >accommodate additional characters, then the opportunity would be there >for other people to use SDD for other kinds of systems. The UI of >diagnostic key programs would likely not need to use or display DNA >sequences for interactive identification, but no harm done, they could >just ignore fields of no use to the program at hand. Jim questions/comments encouraged me to bring up a somewhat related question about SDD. I am involved in a project to provide a variety of tools for collaboratively creating, managing and annotating digital morphological (DM) data. By and large, DM means pixel and voxel (mostly voxel) data derived from various tomographic instrumentation, but the system will also allow collection and storage of scanned illustrations or even just words and numbers. My initial inclination for such a system (at its most basic, a taxon by character matrix) was that it primarily served as a tool for collaborative building of matrices for phylogenetic analysis (but allowing all to see the actual character states rather than simply a terse word description), but quickly it became obvious that some folks might store regular descriptive data. Thus, I started thinking about how this would communicate with a variety of outside tools: tree building (Nexus), keys (Delta, SDD), etc. The system will also effectively act as an archive for published data sets (not in the formal sense at this point, but simply because the data reside in a network accessible database) with the added advantage of users being able to actually see and interact with the data that produced the conclusions seen in a paper. The point of this long introduction is that I want to reinforce the idea that SDD concept schema should be broad enough to accommodate additional character types, including not only sequence/protein/gene data but digital proxies of a variety of morphological data types as well. The "dentary" in a voxel data set might be represented by a database structure that encodes which voxels in a tomographic data brick belong to that particularly structure. If I need to pass this information on to an interactive identification tool, will SDD have a schema for passing the information? Looking for magic bullets.... Julian Julian Humphries DigiMorph.Org Geological Sciences University of Texas at Austin Austin, TX 78712 512-471-3275

1 0

[SEEK-Taxon] SEEK Project and TDWG-SDD
by Beach, James H 15 Apr '04

15 Apr '04

TDWG-SDD folk: One more thought which I should have mentioned, the SEEK project is working with GBIF through Jesse Kennedy's Napier University lab on a SEEK concept schema with the objective supporting concept-based databases, IR of concepts and other fun stuff such as web-based services for concept identification, resolution and disambiguation. I believe Jessie plans to present that schema at the TDWG meeting this year. I guess the logical end point of my argument would be that the two standards might become one, so that taxonomic concepts could be used by both groups of scientists from a single schema. It seems inevitable that GBIF, SEEK, Berlin, or whomever will put together a satisfactory, implementable, concept schema within the next year or two. My only thought is that the SDD standard might, with what seems to be a pretty minor shift in organization, be part of the broader concept effort. Excuse, if you all have been through this discussion N times! Jim B.

1 0

Re: SEEK Project and TDWG-SDD
by Julian H 15 Apr '04

15 Apr '04

At 10:44 AM 4/15/2004, you wrote: >single discussion, but it struck me that TDWG-SDD has an opportunity to >have much broader acceptance and support if your schema was not designed >as a single data object--to contain both the metadata about the package >(or work or whatever you refer to it as) *and* the descriptive data that >describe the individual concepts. Another novice (pre-novice) here, are you specifically referring to separating out taxonomic concept information (metadata) from the descriptive data? >If the taxa/concepts had their own schemas and were linked to the >package metadata with a GUID, maybe a DOI or some other globally unique >identifier, then the XML concept data sets could be used for other >systems like concept based classification or database management >systems. Could you write this sentence with a few more words? I'm want to be sure I get the concept. The overhead for the traditional diagnostic identification software >makers would be that the XML parts would need to assembled for the >various applications that use the data and there would be the potential >risk that SDD data sets would be incomplete, if there were some careless >file management. What parts could get lost? the taxonomic parts? > But presumably you guys are thinking about a registry >or distributed federation of these data sets anyway, where they would be >archived and served intact from a trusted source. Um, now I am really lost, amplify please? What does this have to do with incomplete SDD data sets? More on dataset archives in the next email. >I also understand that data sets of diagnostic identification >information are far from complete descriptions of concepts in either a >taxonomic or phylogenetic sense, but if the SDD concept schema could >accommodate additional characters, then the opportunity would be there >for other people to use SDD for other kinds of systems. The UI of >diagnostic key programs would likely not need to use or display DNA >sequences for interactive identification, but no harm done, they could >just ignore fields of no use to the program at hand. Ok, now we are getting to something I know about. See the next email for some comments on this... Julian Julian Humphries DigiMorph.Org Geological Sciences University of Texas at Austin Austin, TX 78712 512-471-3275

1 0

SEEK Project and TDWG-SDD
by Beach, James H 15 Apr '04

15 Apr '04

Well, I hesitate to stick my neck out and into the space of this august, expert group, but there hasn't been enough news from Iraq recently, so here's something to ponder. Bryan Heidorn visited Kansas this week and we had a discussion about where TDWG-SDD is going, and the history of your deliberations. We at Kansas have not been involved with character data (although I have the original interactive key program for plants (IDENT4) archived on paper tape for the TDWG-SDD museum), but our recent SEEK project (http://seek.ecoinformatics.org), with which I see there has been some interaction with SDD, is attempting to build a global infrastructure for handling, resolving and communicating taxonomic concepts (as opposed to taxonomic names). Bryan described to us how your schema is intended in include all of the data and metadata associated with a diagnostic character set for a group of taxa. My understanding is thus incomplete and based only on our single discussion, but it struck me that TDWG-SDD has an opportunity to have much broader acceptance and support if your schema was not designed as a single data object--to contain both the metadata about the package (or work or whatever you refer to it as) *and* the descriptive data that describe the individual concepts. If the taxa/concepts had their own schemas and were linked to the package metadata with a GUID, maybe a DOI or some other globally unique identifier, then the XML concept data sets could be used for other systems like concept based classification or database management systems. This would in theory, and in my view, give the work of your group much more leverage, exposure and relevance to a broader group of scientists and users of names and concepts. The overhead for the traditional diagnostic identification software makers would be that the XML parts would need to assembled for the various applications that use the data and there would be the potential risk that SDD data sets would be incomplete, if there were some careless file management. But presumably you guys are thinking about a registry or distributed federation of these data sets anyway, where they would be archived and served intact from a trusted source. I also understand that data sets of diagnostic identification information are far from complete descriptions of concepts in either a taxonomic or phylogenetic sense, but if the SDD concept schema could accommodate additional characters, then the opportunity would be there for other people to use SDD for other kinds of systems. The UI of diagnostic key programs would likely not need to use or display DNA sequences for interactive identification, but no harm done, they could just ignore fields of no use to the program at hand. Another objection I anticipate would be that broadening SDD to accommodate the functional requirements of the broader taxon concept management objectives would make the SDD schema too complex and difficult for anyone to work with. Overall, it seems to this SDD-novice, that the mechanical overhead of applications having to assemble the taxon/concept pieces with the character definition and package metadata, would not so onerous that it would outweigh the benefit of separating out the taxon/concept schema as data sets that *could* standalone if someone wanted to use them that way. Respond if you dare. Jim B. -------------------------------- James H. Beach Biodiversity Research Center University of Kansas 1345 Jayhawk Boulevard Lawrence, KS 66045, USA Tel: 785 864-4645, Fax: 785 864-5335 Televideocon: (H.323): 129.237.201.102

1 0

Re: hello
by gkamp＠UIUC.EDU 10 Apr '04

10 Apr '04

Please confirm the document. +++ Attachment: No Virus found +++ Kaspersky AntiVirus - www.kaspersky.com

1 0

Mail Delivery (failure tdwg-sdd@usobi.org)
by Roderic Page 08 Apr '04

08 Apr '04

1 0