- tdwg-content - lists.tdwg.org

Re: TDWG-SDD XML proposals
by Kevin Thiele 01 Nov '00

01 Nov '00

From: Mike Dallwitz <md(a)ENTO.CSIRO.AU> To: <TDWG-SDD(a)USOBI.ORG> Sent: Tuesday, October 31, 2000 1:32 PM Subject: Re: TDWG-SDD XML proposals of Kevin Thiele | There is no mention of character lists in this draft (perhaps it is to come | in later versions). There is currently no way of ensuring that descriptions | are comparative. The intention (at least from where I sit) is to incorporate (optional) character lists later - again, once idrefs are implemented. Cheers - k

1 0

Re: TDWG-SDD XML proposals
by Kevin Thiele 01 Nov '00

01 Nov '00

Jim wrote: > Anyway, it looks as though we are settling on an architecture allows both > discursive descriptions accommodated by 'narrative' elements, and for the > compulsive obsessives, a nested set of 'features' that can have > 'feature_values', 'qualifers' and associated 'narrative'... Right? > > And that the competing architecture of a list of 'feature_values' that can > be present, absent, unknown, doubtful, present by misinterpretation, absent > by misinterpretation, imaginary, etc. has been given the flick? There are two things mixed in here, surely. 1. Yes, I think an architecture that allows both discursive (read "natural language" if you will) and coded (DELTA-like) descriptions is feasible. The principal difference between discursive and coded forms is that one uses idrefs and the other doesn't - so as Bob notes, once the XMLHeads work out how to use idrefs here we will hopefully be able to accommodate the coded form. 2. In either case, there is still a need for a limited set of qualifiers for feature values (ie present, uncertain, present by misinterpretation etc). This will surely simply be another layer of complexity that will be added to the core once it's robust. This is partially implemented already (using two rather curious qualifiers): - <FEATURE name="color"> <FEATURE_VALUE QUALIFIER="highly">varies between colonies</FEATURE_VALUE> <FEATURE_VALUE QUALIFIER="maybe">concolorous yellow-brown</FEATURE_VALUE> <FEATURE_VALUE>bicolored with darker head</FEATURE_VALUE> <FEATURE_VALUE>concolorous brown</FEATURE_VALUE> </FEATURE> Presumably, Bob's comments: >As we represent it, TDD0.3 has only a few classes of objects: >FEATURE a complex type with a name attribute and containing a string-based FEATURE_VALUE, >NARRATIVEs and, recursively, other FEATUREs. >DESCRIPTION a container of FEATUREs and NARRATIVES. >NARRATIVE a string-based type suitable for extension to more complex markup. >Our goal was the production and application of a Schema for TDD0.3, and consequently in the >sample applications we have forced some things into the model which are likely, ultimately, >to have their own Schema. The notable examples are publishing artifacts such as markup for >literature references, and some common scientific vocabulary deserving its own tagging standards. means that many things will eventually be moved out of the rather featureless FEATURE and DESCRIPTION objects into more purpose-built structures. Cheers - k

1 0

Re: TDWG-SDD XML proposals
by Robert A. (Bob) Morris 01 Nov '00

01 Nov '00

Kevin Thiele writes: > Date: Wed, 1 Nov 2000 16:32:28 +1100 > From: Kevin Thiele <kevin.thiele(a)PI.CSIRO.AU> > To: TDWG-SDD(a)usobi.org > Subject: Re: TDWG-SDD XML proposals > > Jim wrote: > > > Anyway, it looks as though we are settling on an architecture allows both > > discursive descriptions accommodated by 'narrative' elements, and for the > > compulsive obsessives, a nested set of 'features' that can have > > 'feature_values', 'qualifers' and associated 'narrative'... Right? > > > > And that the competing architecture of a list of 'feature_values' that > can > > be present, absent, unknown, doubtful, present by misinterpretation, > absent > > by misinterpretation, imaginary, etc. has been given the flick? > > There are two things mixed in here, surely. > > 1. Yes, I think an architecture that allows both discursive (read "natural > language" if you will) and coded (DELTA-like) descriptions is feasible. The > principal difference between discursive and coded forms is that one uses > idrefs and the other doesn't - so as Bob notes, once the XMLHeads work out > how to use idrefs here we will hopefully be able to accommodate the coded > form. Well, only part of this intersects the distinction that we are working against, though the non-intersection is certainly important one. The distinction we are presently thinking about is purely syntactic. It is whether the XML data representation is tree structured (by a non-linear tree) and without idref's or is linearly structured and with idrefs. In fact all trees can be flattened into lists if by nothing else than adding references to the mix. It's easy to be convinced of this: trees can be represented in the conventional linear memory supplied with most computers, so it is clearly possible (and not that hard). What we want to get right is not whether this can be done---since we know it can---but specifically whether XML's reference mechanisms will capture the requirements of the intended stakeholders in the enterprise. We hope it does, because there is a BIG advantage to using XML: there is an abundance of tools, protocols, databses and services available to those who adopt it. The question of discursive in the sense of natural langauge doesn't directly enter our present efforts, though we've been thinking a little about it. Gregor Hegedorn has implemented an approach in which his database is the source of the natural language syntax in that it stores verbs that a given description can access to produce something that produces---at worst---bemused smiles from a human reader. This may be a reasonable for discursive representations in XML too but we didn't attempt to capture it. Rather we forced some of our sample into our model by changing the English grammar a little. For example, we text whose original was part of a diaganosis whose discourse was Workers: scapes and tibiae lack erect hair we marked as <FEATURE name="scapes and tibiae erect hair"> <FEATURE_VALUE>lack</FEATURE_VALUE> </FEATURE> Naive discursive rendering of this would probably emerge as scapes and tibiae erect hair lack. Of course, Gregor might be happy about that sentence order. :-) For me, what is lurking under the covers here is this: I don't think that high quality human readability of the naked markup is that important. I think it is the job of application software to render markup palatable to humans if that is the problem, or to other software if that is the problem. > > 2. In either case, there is still a need for a limited set of qualifiers for > feature values (ie present, uncertain, present by misinterpretation etc). > This will surely simply be another layer of complexity that will be added to > the core once it's robust. > > This is partially implemented already (using two rather curious qualifiers): > - <FEATURE name="color"> > <FEATURE_VALUE QUALIFIER="highly">varies between colonies</FEATURE_VALUE> > <FEATURE_VALUE QUALIFIER="maybe">concolorous yellow-brown</FEATURE_VALUE> > <FEATURE_VALUE>bicolored with darker head</FEATURE_VALUE> > <FEATURE_VALUE>concolorous brown</FEATURE_VALUE> > </FEATURE> We extracted "highly" and "maybe" from the original. This raises an interesting point. It may well be good to recommend a specific set of qualifiers. That allows more intelligent applications. For example, (and neglecting that "highly variable" and "frequently variable" are probably not acceptable as nearly equivalent ) it might be easier to guide identification software if "frequently" is uniformly used where appropriate, thus perhaps identifying some initially more (or less) important feature. However, (a).An author might not be completely satisfied with that qualifier and (b) an author may be satisfied but prefer something else in natural language discourse. Probably this entails recommending a mechanism for specifying alternatives to the 'official' qualifier. Those mechanisms could guide applications that were aware of them. For example <FEATURE_VALUE QUALIFIER="frequently" RENDER_QUALIFIER_AS="highly"> > > Presumably, Bob's comments: > >As we represent it, TDD0.3 has only a few classes of objects: > >FEATURE a complex type with a name attribute and containing a string-based > FEATURE_VALUE, > >NARRATIVEs and, recursively, other FEATUREs. > >DESCRIPTION a container of FEATUREs and NARRATIVES. > >NARRATIVE a string-based type suitable for extension to more complex > markup. > > >Our goal was the production and application of a Schema for TDD0.3, and > consequently in the > >sample applications we have forced some things into the model which are > likely, ultimately, > >to have their own Schema. The notable examples are publishing artifacts > such as markup for > >literature references, and some common scientific vocabulary deserving its > own tagging standards. > > means that many things will eventually be moved out of the rather > featureless FEATURE and DESCRIPTION objects into more purpose-built > structures. > > Cheers - k >

1 0

Re: TDWG-SDD XML proposals
by Robert A. (Bob) Morris 01 Nov '00

01 Nov '00

Kevin Thiele writes: > Date: Wed, 1 Nov 2000 17:21:45 +1100 > From: Kevin Thiele <kevin.thiele(a)PI.CSIRO.AU> > To: TDWG-SDD(a)usobi.org > Subject: Re: TDWG-SDD XML proposals > > From: Mike Dallwitz <md(a)ENTO.CSIRO.AU> > To: <TDWG-SDD(a)USOBI.ORG> > Sent: Tuesday, October 31, 2000 1:32 PM > Subject: Re: TDWG-SDD XML proposals of Kevin Thiele > > > | There is no mention of character lists in this draft (perhaps it is to > come > | in later versions). There is currently no way of ensuring that > descriptions > | are comparative. > > The intention (at least from where I sit) is to incorporate (optional) > character lists later - again, once idrefs are implemented. Our intention exactly. But not *too much* later, given how important interoperability with the large amount of list based data there is. In fact, it will be the next thing we work on. I don't think that a start (including translations back and forth from lists to trees) will be very difficult. I do suspect that discussion of details will end up being extensive. At W3C this kind of thing often goes through 7-8 rounds of comment and revision before a recommendation appears. People may wish to see Sec 6.2 of the W3C Process Document to see how they proceed to bring something through to the status of Recommendation. (They leave "Standard" to standards bodies). > > Cheers - k >

1 0

Re: TDWG-SDD XML proposals of Kevin Thiele
by Mike Dallwitz 31 Oct '00

31 Oct '00

> The material at http://www.cs.umb.edu/efg/ThieleDraft/thiele_0_3.html > illustrates our efforts to draft an XML Schema from Thiele's draft > proposal version 0.3 ("TDD0.3") There is no mention of character lists in this draft (perhaps it is to come in later versions). There is currently no way of ensuring that descriptions are comparative. -- Mike Dallwitz CSIRO Entomology, GPO Box 1700, Canberra ACT 2601, Australia Phone: +61 2 6246 4075 Fax: +61 2 6246 4000 Email: md(a)ento.csiro.au Internet: biodiversity.uno.edu/delta/

1 0

Re: TDWG-SDD XML proposals of Kevin Thiele
by Eric Zurcher 31 Oct '00

31 Oct '00

At 23:57 30-10-2000 +1100, Jim Croft wrote, in reply to Bob Morris: >>The material at >>http://www.cs.umb.edu/efg/ThieleDraft/thiele_0_3.html illustrates our >>efforts to draft an XML Schema from Thiele's draft proposal version >>0.3 ("TDD0.3") and illustrate its utility with simple >>applications manipulating a taxonomic treatment. > >No-one seems to be commenting on this, at least on the list... is it all >too scary? With Halloween just around the corner, do these guys deserve to >be tricked or treated? Well, I certainly hope that responses aren't limited to those of us having mailboxes at pi.csiro.au (that is, Kevin, Jim, and myself). I applaud the effort Bob Morris and Jun Wan have gone to it constructing this example. They have provided a good demonstration of the power of XML and XSLT to hold information and rapidly render it in a variety of forms. I suspect most of us find that a concrete working example of this sort is a lot easier to evaluate than is a relatively abstract discussion of standards. >This is a great attempt at implementation and surely demonstrates that we >are on the right track... doesn't it? It may well be the right track (depending on the desired destination), but there's still a lot of ground to cover before arriving at the final stop... One thing that might be helpful would be for this group to come up with a short set of basic design goals that indicate what our objectives really are. My understanding is that the developers of XML itself found such a set of goals to be very useful. Their list of goals was (and still is) to quote from the W3C Recommendation for XML 1.0: "The design goals for XML are: XML shall be straightforwardly usable over the Internet. XML shall support a wide variety of applications. XML shall be compatible with SGML. It shall be easy to write programs which process XML documents. The number of optional features in XML is to be kept to the absolute minimum, ideally zero. XML documents should be human-legible and reasonably clear. The XML design should be prepared quickly. The design of XML shall be formal and concise. XML documents shall be easy to create. Terseness in XML markup is of minimal importance." Can we develop a similar set of goals and principles (perhaps even using some of the XML principles as a basis)? One other point - Bob Morris said: >My own view is that there should actually be a single standard >representation with a bunch of ways to transform between it and other >desired representations. The advantage of this is that you need fewer >translators to go between representations A and B if everything has a >bidirectional representation with a common one, X. The disadvantage is >that you change X at the risk of breaking X<->A, thereby breaking >A<->B when neither A nor B changed. The goal of a single standard representation readily translatable into other representations was, of course, the original goal of DELTA, and its advantages are clear. But perhaps the biggest challenge with any such standard is that it needs to be very comprehensive and flexible - ideally, it should be able to store everything that all of the "subsidiary" formats is capable of representing, and do so in such a way that X<->Y translations are as lossless as possible. Eric Zurcher CSIRO Livestock Industries Canberra, ACT Australia E-mail: Eric.Zurcher(a)pi.csiro.au

1 0

Re: TDWG-SDD XML proposals of Kevin Thiele
by Jim Croft 30 Oct '00

30 Oct '00

>The material at >http://www.cs.umb.edu/efg/ThieleDraft/thiele_0_3.html illustrates our >efforts to draft an XML Schema from Thiele's draft proposal version >0.3 ("TDD0.3") and illustrate its utility with simple >applications manipulating a taxonomic treatment. No-one seems to be commenting on this, at least on the list... is it all too scary? With Halloween just around the corner, do these guys deserve to be tricked or treated? Unless people are commenting directly to Kevin/Bob/Jun, I would imagine they would be feeling pretty disappointed so far... I have been through this a number of times, still trying to come to grips with the various options that have been used from the draft specs of a few weeks ago... and why... This is a great attempt at implementation and surely demonstrates that we are on the right track... doesn't it? Yes, the format does seem a bit verbose, but from what I gather Bob and Jun are saying, let the software, applications and databases worry about that... Personally I prefer something like: <taxonomy> <rank="species" value="alfari"/> <rank="genus" value="Azteca"/> <author value="Emery"/> <year value="1893"/> </taxonomy> to something like: <DESCRIPTION name="Taxonomic Information"> <FEATURE name="Species Name"> <FEATURE_VALUE>alfari</FEATURE_VALUE> </FEATURE> <FEATURE name="Genus Name"> <FEATURE_VALUE>Azteca</FEATURE_VALUE> </FEATURE> <FEATURE name="Namer"> <FEATURE_VALUE>Emery</FEATURE_VALUE> </FEATURE> <FEATURE name="Name Year"> <FEATURE_VALUE>1893</FEATURE_VALUE> </FEATURE> </DESCRIPTION> but if we never have to read it in this format, I do not suppose it really matters... Anyway, it looks as though we are settling on an architecture allows both discursive descriptions accommodated by 'narrative' elements, and for the compulsive obsessives, a nested set of 'features' that can have 'feature_values', 'qualifers' and associated 'narrative'... Right? And that the competing architecture of a list of 'feature_values' that can be present, absent, unknown, doubtful, present by misinterpretation, absent by misinterpretation, imaginary, etc. has been given the flick? Or having coded the data in this manner, is this distinction in data representation employed by different interactive identification products, not really relevant or meaningful? Sorry about the rambling - just trying to come to grips with what has been done here... I am pretty sure it is good stuff once I work out what it all means... One vague background concern I have is with things like: <FEATURE name="Page Author"> <FEATURE name="name"> <FEATURE_VALUE>John T. Longino</FEATURE_VALUE> </FEATURE> <FEATURE name="efg address"> <FEATURE_VALUE>The Evergreen State College, Olympia WA 98505 USA</FEATURE_VALUE> </FEATURE> <FEATURE name="email"> <FEATURE_VALUE>longinoj(a)elwha.evergreen.edu</FEATURE_VALUE> </FEATURE> </FEATURE> which seems on the surface to correspond to ye olde library cataloguing metadata stuff. Is there a wheel here we do not need to reinvent? Related to this there are a number of projects around to place using XML to mark up and display descriptive data and taxonomic treatments and they seem to inventing rafts of 'feature' names for what appear to be the same things in terms of blocks onf information. At the higher and metadata level, is it too early to talk about rationalizing these? Being very wary here, not wanting to open the Pandora's character-list box of worms again... jim

1 0

Re: TDWG-SDD XML proposals of Kevin Thiele
by Robert A. (Bob) Morris 30 Oct '00

30 Oct '00

Jim Croft writes: > Date: Mon, 30 Oct 2000 23:57:44 +1100 > From: Jim Croft <jrc(a)anbg.gov.au> > To: TDWG-SDD(a)usobi.org > Subject: Re: TDWG-SDD XML proposals of Kevin Thiele > > >The material at > >http://www.cs.umb.edu/efg/ThieleDraft/thiele_0_3.html illustrates our > >efforts to draft an XML Schema from Thiele's draft proposal version > >0.3 ("TDD0.3") and illustrate its utility with simple > >applications manipulating a taxonomic treatment. > > No-one seems to be commenting on this, at least on the list... is it all > too scary? With Halloween just around the corner, do these guys deserve to > be tricked or treated? > > Unless people are commenting directly to Kevin/Bob/Jun, I would imagine > they would be feeling pretty disappointed so far... > > I have been through this a number of times, still trying to come to grips > with the various options that have been used from the draft specs of a few > weeks ago... and why... > > This is a great attempt at implementation and surely demonstrates that we > are on the right track... doesn't it? > > Yes, the format does seem a bit verbose, but from what I gather Bob and Jun > are saying, let the software, applications and databases worry about that... > > Personally I prefer something like: > > <taxonomy> > <rank="species" value="alfari"/> > <rank="genus" value="Azteca"/> > <author value="Emery"/> > <year value="1893"/> > </taxonomy> > > to something like: > > <DESCRIPTION name="Taxonomic Information"> > <FEATURE name="Species Name"> > <FEATURE_VALUE>alfari</FEATURE_VALUE> > </FEATURE> > <FEATURE name="Genus Name"> > <FEATURE_VALUE>Azteca</FEATURE_VALUE> > </FEATURE> > <FEATURE name="Namer"> > <FEATURE_VALUE>Emery</FEATURE_VALUE> > </FEATURE> > <FEATURE name="Name Year"> > <FEATURE_VALUE>1893</FEATURE_VALUE> > </FEATURE> > </DESCRIPTION> > > but if we never have to read it in this format, I do not suppose it really > matters... > > Anyway, it looks as though we are settling on an architecture allows both > discursive descriptions accommodated by 'narrative' elements, and for the > compulsive obsessives, a nested set of 'features' that can have > 'feature_values', 'qualifers' and associated 'narrative'... Right? > > And that the competing architecture of a list of 'feature_values' that can > be present, absent, unknown, doubtful, present by misinterpretation, absent > by misinterpretation, imaginary, etc. has been given the flick? Not really. We believe that it is pretty straightforward to translate between this kind of format and the one more spiritually akin to XML and will illustrate that Real Soon Now. The biggest thing we need to think about is exactly how the XML id reference mechanisms should be used to implement things as in Kevin's examples. My own view is that there should actually be a single standard representation with a bunch of ways to transform between it and other desired representations. The advantage of this is that you need fewer translators to go between representations A and B if everything has a bidirectional representation with a common one, X. The disadvantage is that you change X at the risk of breaking X<->A, thereby breaking A<->B when neither A nor B changed. That said, I think that this issue should not have lots of philosophical discussion until we, or somebody else, demonstrates the conversion between list-like formats and tree-like formats. > > Or having coded the data in this manner, is this distinction in data > representation employed by different interactive identification products, > not really relevant or meaningful? > > Sorry about the rambling - just trying to come to grips with what has been > done here... I am pretty sure it is good stuff once I work out what it all > means... > > One vague background concern I have is with things like: > > <FEATURE name="Page Author"> > <FEATURE name="name"> > <FEATURE_VALUE>John T. Longino</FEATURE_VALUE> > </FEATURE> > <FEATURE name="efg address"> > <FEATURE_VALUE>The Evergreen State College, Olympia WA 98505 > USA</FEATURE_VALUE> > </FEATURE> > <FEATURE name="email"> > <FEATURE_VALUE>longinoj(a)elwha.evergreen.edu</FEATURE_VALUE> > </FEATURE> > </FEATURE> > > which seems on the surface to correspond to ye olde library cataloguing > metadata stuff. Is there a wheel here we do not need to reinvent? Absolutely. Per our remarks in the web page, we mainly did this out of time constraints. For metadata, there is no doubt that the Dublin Core and its extension the Darwin Core should be looked at. For bibliographic things, the American Publishers Association has an SGML DTD which is likely to be XML by now, etc. I'll write some references to those for people to consider. > > Related to this there are a number of projects around to place using XML to > mark up and display descriptive data and taxonomic treatments and they seem > to inventing rafts of 'feature' names for what appear to be the same things > in terms of blocks onf information. At the higher and metadata level, is > it too early to talk about rationalizing these? Being very wary here, not > wanting to open the Pandora's character-list box of worms again... Can we start assembling a list? > > jim > Bob Morris

1 0

Re: TDWG-SDD XML proposals of Kevin Thiele
by Kevin Thiele 30 Oct '00

30 Oct '00

Bob's put a draft SDD proposal up for comment. Note that this is a skeleton only - we're trying to work out at the moment how to capture the base elements. Other issues discussed in earlier SDD drafts and through this list - such as data attribution, collation, inheritance etc - will be incorporated if this stuff seems to work so far. Cheers - k

1 0

TDWG-SDD XML proposals of Kevin Thiele
by Robert A. (Bob) Morris 28 Oct '00

28 Oct '00

[This is bcc'd to several mailing lists. Apologies for duplicates you may receive.] The Taxonomic Working Group subgroup on the Structure of Descriptive Data (TDWG-SDD) of the International Union of Biological Sciences maintains an email discussion list on which there has been recent discussion of proposals of Kevin Thiele to formulate XML representations of a minimal controlled vocabulary for taxonomic descriptive data. The material at http://www.cs.umb.edu/efg/ThieleDraft/thiele_0_3.html illustrates our efforts to draft an XML Schema from Thiele's draft proposal version 0.3 ("TDD0.3") and illustrate its utility with simple applications manipulating a taxonomic treatment. Some of the pieces are best viewed with MSIE5, but any browser should do. We'll be talking about this work at the TDWG meeting in Frankfurt November 10-12. Bob Morris Jun Wan

1 0