- tdwg-content - lists.tdwg.org

Re: RQTS : tag order
by Gregor Hagedorn 13 Dec '99

13 Dec '99

Bob Morris wrote: > Here's a proposed requirement for data exchange, brought home by one > of Leigh's examples: > > <RQT> It should be possible from the data itself, perhaps together > with external resources to which it refers, for software to determine > whether the order matters for any collection of tags in the > data.</RQT> I agree, especially that the software should be able to determine it. Regarding XML, tags, and DTDs, I believe, the assumption is that order does not matter, wherease an order can be imposed on attributes within a tag, correct? > One answer is "order always matters". That would be regrettable. For > example, being able to detect that some list is unordered can make it > easier to know whether two data streams represent the same > object. (That's probably a deeper desideratum.) Does order matters for our problem? Where would it matter, any examples from descriptive data? In fact, thinking in my own relational information model, we have a problem here: Some parts of the structure will be relating to records, i.e. instances of an entity type. In my thinking, the order of items to be described, and the order of characters used to perform the description should NOT matter. It is a plausible user requirement to report any of them in any order. That does not mean that characters or states do not have an order, but this order is secondary, and multiple orders are desirable. Being ordered should not be a requirement of the exchange format. However, there are other data elements that refer to attributes of entities. Ordering them makes some parsing operations easier. However, what fundamental advantages would it have to impose an order on them? Finally, some item are ordered, e.g. it will be desirable to impose an order on multiple states in a descriptions. Written in (loose) DELTA: 1,1/<rarely>2 -> Flowers blue or rarely white may be different from 1,2/1<rarely> -> Flowers white or rarely blue However, I believe that the DELTA method of using a text-sequence is not necessarily the best solution. Alternatively, a data element may be introduced to store the ordering information. For example, DeltaAccess handles this problem with the numeric SEQ attribute of the DESCR entity. Gregor ---------------------------------------------------------- Gregor Hagedorn G.Hagedorn(a)bba.de Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

My summary of "Why use XML"
by Gregor Hagedorn 13 Dec '99

13 Dec '99

Kevin brought up the question of advantages of XML. Somewhat belated, because I have been away for 10 days, here are some comments, trying to summarize the discussion as I see it. 1. I agree entirely with Noel that the discussions about XML on this list are intended to _explore_ the advantages of XML, not to settle on it. XML is well defined to be used in the context of text documents, but my impression is that for data that contain more structure, the XML variants are right now being developed (for example XML-Schema, XML-RDF). So it will be good to wait a moment before making decisions. And there is much to do regarding the information model itself! I hope that we will have more discussions about this. 2. I think Leigh Dodds did a good job in listing the advantages in his XDelta document. Generally I think that using standards and standard tools is a good idea. However, most certainly, XML will not be an "ultimate standard", as Jean-Marc thinks it is. XML will be certainly replaced by improved mechanisms. My feeling is that it is sufficiently relevant a step that it will be supported in a backwards compatible way even in 50 years. Of course, nobody can predict this. 3. Regarding the issue of size of data set, verbosity, and data compression: Some comments, esp. when it comes to "using it on PalmTops etc." confuse the issue of an exchange format with a native data format. Esp. given that XSLT will make it very easy to convert data, any program will most likely have a native, usually binary format that is optimized for operation speed and/or file size. I think we should not consider size any factor of an _exchange_ format. This format should be as robust and compatible for the future as possible. Many of us are developers of programs and tend to underestimate the effort that goes into the development of data sets. Our programs will certainly be replaced by better ones, but if the data can be maintained, revised, and improved as it is done for centuries with printed publications, I believe we will make better progress in biodiversity research. The data sets are the gold, and they should be preserved in a way that remains readable in 50 years. Because of this, however, using compression algorithms and then write it on a CD may be a bad idea for long term storage. Imagine having to decompress something that was compressed with some program on a Apple II machine... Gregor ---------------------------------------------------------- Gregor Hagedorn G.Hagedorn(a)bba.de Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: RQTS : tag order
by Robert A. (Bob) Morris 13 Dec '99

13 Dec '99

Gregor Hagedorn writes: > Date: Mon, 13 Dec 1999 16:21:15 +0100 > From: Gregor Hagedorn <G.Hagedorn(a)bba.de> > To: TDWG-SDD(a)usobi.org > Subject: Re: RQTS : tag order > > Bob Morris wrote: > > > Here's a proposed requirement for data exchange, brought home by one > > of Leigh's examples: > > > > <RQT> It should be possible from the data itself, perhaps together > > with external resources to which it refers, for software to determine > > whether the order matters for any collection of tags in the > > data.</RQT> > > I agree, especially that the software should be able to determine it. > Regarding XML, tags, and DTDs, I believe, the assumption is that > order does not matter, wherease an order can be imposed on attributes > within a tag, correct? No, it is exactly the opposite. A DTD imposes an order on tags, but attributes are unordered. However, the November 5 working draft of XML Schema provides mechanisms for settling tag order questions. XML Schema is a language for specifying tag constraints in XML itself, rather than in the DTD language which is not itself a special case of XML. Among other things, it provides strong data typing, including the ability to restrict the range of data, whereas in a DTD the only primitive data type is a string. It is slightly plausible that XML Schema would actually provide a good data modeling language independent of whether one wished to use it as an interchange language or as the native data store. Bob Morris

1 0

Re: (XML)sharing files between programs without data loss
by Jean-Marc Vanel 12 Dec '99

12 Dec '99

Hello Leigh Dodds is absolutely right in is "no silver bullet" message. What I had in mind is an answer to the problem raised by "Susan B. Farmer" <sfarmer(a)GOLDSWORD.COM> ... there was no way to take *this* file back into DELTA. But more than that; all of the continuous data in the original file (aka measurements) are now gone and will have to be added back into the DELTA data set by hand. :-( Why not extend the current standards that if there are data types that their particular application doesn't recognize, they stay around -- just ignored. So for the programmer it implies that the DOM model should be kept throughout the application program, and not be rewriten from scratch when the user pushes "save". About Leigh Dodds' example: <application-B-tag references="8"> I referred to standard chart X to finally decide on shapes. </application-B-tag> The 'references' attribute is like a pointer in C, or a foreign key in a database. The example's application B could have put its tag inside the concerned <character> element. In this case application A would logically destroy the comment of B together with with the character. Or another solution would be for the 'references' attribute to be an IDREF (XML's reference to a document-unique ID); in this way the DOM implementation would probably forbid the deletion of the element being referenced. Cheers JMV

1 0

TDWG room
by P. F. Stevens 10 Dec '99

10 Dec '99

Yes, the room was put on my credit card. So, if TDWG covered the room by giving you the cost, then you can foward what we gave you to me, and I can.... I have been silent the last couple of weeks about data exchange mainly because it is the end of term - I don't even know if there is anything I want to say, since I have been letting the messages accumulate, but I will try and go through them some time next week. p.

1 0

Re: (GEN) CD's for Xmas
by Joseph H. Kirkbride, Jr. 10 Dec '99

10 Dec '99

You should consider our CD-ROM on legume fruits and seeds: http://www.netins.net/showcase/alurir/seeds.html Legume (Fabaceae) Fruits and Seed (Interactive CD-ROM), by Joseph H. Kirkbride, Jr., Charles R. (Bob) Gunn, Anna L. Weitzman, and Michael J. Dallwitz Joseph H. Kirkbride, Jr. USDA, Agricultural Research Service Systematic Botany and Mycology Laboratory Room 304, Building 011A, BARC-West Beltsville, Maryland 20705-2350 USA Voice telephone: 301-504-9447 FAX: 301-504-5810 Internet: jkirkbri(a)asrr.arsusda.gov On Thu, 9 Dec 1999, Jean-Marc Vanel wrote: > Hello > > I'm pressed to tell my wishes for Christmas. > Do you have suggestions of CD's about botany, zoology, or ecology ? > > Thanks > JMV >

1 0

(XML) Erratum
by Jean-Marc Vanel 09 Dec '99

09 Dec '99

Hello from Paris I'm sorry that have writen "shearing" for "sharing"; it's funny, and somewhat appropriate, but wasn't meant. (cf my mail "(XML)shearing files between programs without data loss"). I'll answer in detail, it's important. Cheers JMV

1 0

(GEN) CD's for Xmas
by Jean-Marc Vanel 09 Dec '99

09 Dec '99

Hello I'm pressed to tell my wishes for Christmas. Do you have suggestions of CD's about botany, zoology, or ecology ? Thanks JMV

1 0

(GEN) Lexicons
by Kevin Thiele 09 Dec '99

09 Dec '99

There is a subtext running in this discussion - whether part of our scope is the creation of lexicons or standard name-spaces - that to me is causing confusion. For instance: >>>From Leigh: >Restricting states to particular options depending on the property >in question (e.g. leaf and/or wing shape) leads back to the >prior discussion on accepted standards for character description. > >Defining, and agreeing upon these standard notations/descriptions are >A FUNDAMENTAL PART (my caps) of specifying this new format, and one that isn't >solved simply by deciding to use XML (for example). Its part of the >fundamental design and modelling, and is therefore something >that should be addressed early on. But from Leigh again: >I'd say that the DELTA approach - of avoiding domain (i.e. zoology, >virology, etc) specific notations in the format has worked well. >And I think this is the level that any initial work should be pitched at. >i.e. the data format should encode taxonomic *data* - just as DELTA >does. Any domain specific schema can be layered on top of this, or >include it. Begin with capturing the relevant data just as DELTA does, >and then progress from there. So do we or don't we? Am I misinterpreting these that they seem to say opposite things? >>>From Gregor: >It is true: Morphological structures may have containment >hierarchies, but I believe that these depent strongly on the >viewpoint of the author or user. > >EXAMPLE 2: Stuff can be in-between: The inflorescence contains part >of stem, part of leaves, and all flowers. Which leaves are part of >inflorescence and thus called bracts, and which aren't is often a >matter of taste, school, country... > >Thus: there are multiple concurrent or competing hierarchies, which >may overlap. The only problem with competing hierarchies is if we are trying to standardise and resolve the conflicts. If every worker resolves for their own project what to call bracts, this is not a problem for us. >>>From Jean-Marc: >We are designing XML vocabularies for the description of biological >species. Are we? I thought we were designing a format by which such a vocabulary can be represented. For the record, all the current systems (DELTA, LucID, NEXUS etc) enforce nothing lexically, they merely enforce a particular way of representing data. Two data sets for similar groups of plants may contain entirely different characters, or the same characters worded in different ways, or the same characters resolved into states in different ways, or (occasionally) identical characters. Comparing and combining datasets automatically is thus impossible. This seems such a shame, but is it perhaps unavoidable? Thus, if we are designing vocabularies, we are going a long way beyond what's been attempted before. Personally, I think designing domain-specific vocabularies will never work, unless the domain is the individual worker or group of collaborating workers. The popularity of lexicons is the old seductive universalism again. Great idea, but... There are two problems. Firstly, there are (broadly) two types of characters used in descriptions (and keys) - lets call them comparative and diagnostic characters. Comparative characters are the fairly general ones - e.g. leaf shape, ovary position - the sorts of characters that one would aim to describe consistently for all taxa in a monograph. Diagnostic characters are special characters that are useful for separating two or more taxa (of course, sometimes fairly general characters are diagnostic, but not always). A real example of a diagnostic character (from Synaphaea: Proteaceae): Ovary with an apical ring of translucent glands......S. bifurcata Ovary without glands.................................S. oulopha Clearly, no generalised lexicon or name-space will allow for capture of such diagnostic characters. BUT, perhaps we can have a standardised representation for the generalised characters using a lexicon and then use extensibility to allow user-specific diagnostic characters? To some extent, but perhaps not...: I will (foolishly) raise a challenge here that any generalised morphological character that anyone can come up with (in the plant domain) will be entirely inadequate for capturing data for some groups. For example, the most straightforward character I can think of is Leaves present absent But, a diagnostic difference between Discaria pubescens and Discaria nitida (Rhamnaceae) is the degree to which the leaves persist - in both, leaves tend to be absent in the adult plant, but in D. pubescens they are often completely absent while in D. nitida there are usually scattered reduced leaves on younger branchlets. And in Podostemaceae and Utricularia there's no guarantee that a leaf-like part is a leaf, because the conventional differentiation of vegetative parts into leaves/stems doesn't hold. Mother Nature's a tricky old dame, and any character definition will be inadequate to catch her. But do we put up with the inadequacy for the advantages that the universality brings? - if it means we constrain our ability to capture data, then I'd say no. So, I'd like to suggest that we try to develop a standardised data representation, but put no constraints on character definitions whatsoever. Cheers - k Beware the Universe - it bites

1 0

Re: (GEN) Lexicons
by Leigh Dodds 09 Dec '99

09 Dec '99

> There is a subtext running in this discussion - whether part of > our scope is the creation of lexicons or standard name-spaces - that to me is > causing confusion. OK, I'll clarify what I meant by the following two quotes - there is a difference, but it may not be clear.. > >Defining, and agreeing upon these standard notations/descriptions are > >A FUNDAMENTAL PART (my caps) of specifying this new format, and > one that isn't > >solved simply by deciding to use XML (for example). Its part of the > >fundamental design and modelling, and is therefore something > >that should be addressed early on. Here I'm refering standard(ised) ways of recording the data that gets stored in the format - e.g. "leaf shapes will be selected according to the IAPT plain shape chart", "all dates will be in dd-mmm-yyy format". To make the data as interoperable as possible agreeing on standard techniques and measures will help. This is not related to the definition of the format itself - it just defines how its used. My main point being that any data format (new or otherwise) isn't beneficial by itself, unless you know how the data was collected. If that can rely on standard methods then fine. If not then the method of collection needs to be expressed. > >I'd say that the DELTA approach - of avoiding domain (i.e. zoology, > >virology, etc) specific notations in the format has worked well. > >And I think this is the level that any initial work should be pitched at. > >i.e. the data format should encode taxonomic *data* - just as DELTA > >does. Any domain specific schema can be layered on top of this, or > >include it. Begin with capturing the relevant data just as DELTA does, > >and then progress from there. Here I agree with the points you make below - DELTA and the other formats work because they don't attempt to enforce anything lexically. Lets pitch the effort at that level. It would be a mammoth and probably impossible task to capture all possible data usages. >>>From that starting point particular groups can then take that standard and say "when we're recording leaf shapes they will be selected according to the IAPT plain shape chart". This is a layer above a standard data format. Here we're defining its usage. This is where the interoperability problem (sharing data between groups) will be solved. As I say, it may not be possible to make bold statements like "use the IAPT plain shape chart" in every domain, but it may be possible in some domains, or in parts of some domains. This is the layer at which the interesting stuff begins, without it you've just got a file format which happens to look the same for everyone. With agreed usage guidelines, it becomes possible to share data, and make the kind of inferences and extrapolations that we're lead to believe RDF, etc will be able to give us. > So do we or don't we? Am I misinterpreting these that they seem to say > opposite things? Bottom line - I'm talking about two different layers : - a general format for data capture - (possibly domain specific) standardised usage guidelines Does that help? > The only problem with competing hierarchies is if we are trying to > standardise and resolve the conflicts. If every worker resolves for their > own project what to call bracts, this is not a problem for us. Exactly - each worker can resolve their own hiearchy. But if a hierarchy could be agreed between groups then you start to get more use from your data. However any agreement isn't likely to occur between domains (zoology, bryology, virology), but possibly within a domain. > For the record, all the current systems (DELTA, LucID, NEXUS etc) enforce > nothing lexically, they merely enforce a particular way of representing > data. Two data sets for similar groups of plants may contain entirely > different characters, or the same characters worded in different ways, or > the same characters resolved into states in different ways, or > (occasionally) identical characters. Comparing and combining datasets > automatically is thus impossible. This seems such a shame, but is > it perhaps unavoidable? Possibly - but the data format doesn't solve that. Its how you use it. > So, I'd like to suggest that we try to develop a standardised data > representation, but put no constraints on character definitions > whatsoever. We're in agreement here - I'm just suggesting that while there are no constraints codified in the standard, they still need to be expressed (if only in usage guidelines) if you want to be able to share/exchange data meaningfully. Cheers, L.

1 0