December 1999 - tdwg-content

Re: (XML)sharing files between programs without data loss
by Jean-Marc Vanel 12 Dec '99

12 Dec '99

Hello Leigh Dodds is absolutely right in is "no silver bullet" message. What I had in mind is an answer to the problem raised by "Susan B. Farmer" <sfarmer(a)GOLDSWORD.COM> ... there was no way to take *this* file back into DELTA. But more than that; all of the continuous data in the original file (aka measurements) are now gone and will have to be added back into the DELTA data set by hand. :-( Why not extend the current standards that if there are data types that their particular application doesn't recognize, they stay around -- just ignored. So for the programmer it implies that the DOM model should be kept throughout the application program, and not be rewriten from scratch when the user pushes "save". About Leigh Dodds' example: <application-B-tag references="8"> I referred to standard chart X to finally decide on shapes. </application-B-tag> The 'references' attribute is like a pointer in C, or a foreign key in a database. The example's application B could have put its tag inside the concerned <character> element. In this case application A would logically destroy the comment of B together with with the character. Or another solution would be for the 'references' attribute to be an IDREF (XML's reference to a document-unique ID); in this way the DOM implementation would probably forbid the deletion of the element being referenced. Cheers JMV

1 0

TDWG room
by P. F. Stevens 10 Dec '99

10 Dec '99

Yes, the room was put on my credit card. So, if TDWG covered the room by giving you the cost, then you can foward what we gave you to me, and I can.... I have been silent the last couple of weeks about data exchange mainly because it is the end of term - I don't even know if there is anything I want to say, since I have been letting the messages accumulate, but I will try and go through them some time next week. p.

1 0

Re: (GEN) CD's for Xmas
by Joseph H. Kirkbride, Jr. 10 Dec '99

10 Dec '99

You should consider our CD-ROM on legume fruits and seeds: http://www.netins.net/showcase/alurir/seeds.html Legume (Fabaceae) Fruits and Seed (Interactive CD-ROM), by Joseph H. Kirkbride, Jr., Charles R. (Bob) Gunn, Anna L. Weitzman, and Michael J. Dallwitz Joseph H. Kirkbride, Jr. USDA, Agricultural Research Service Systematic Botany and Mycology Laboratory Room 304, Building 011A, BARC-West Beltsville, Maryland 20705-2350 USA Voice telephone: 301-504-9447 FAX: 301-504-5810 Internet: jkirkbri(a)asrr.arsusda.gov On Thu, 9 Dec 1999, Jean-Marc Vanel wrote: > Hello > > I'm pressed to tell my wishes for Christmas. > Do you have suggestions of CD's about botany, zoology, or ecology ? > > Thanks > JMV >

1 0

(XML) Erratum
by Jean-Marc Vanel 09 Dec '99

09 Dec '99

Hello from Paris I'm sorry that have writen "shearing" for "sharing"; it's funny, and somewhat appropriate, but wasn't meant. (cf my mail "(XML)shearing files between programs without data loss"). I'll answer in detail, it's important. Cheers JMV

1 0

(GEN) CD's for Xmas
by Jean-Marc Vanel 09 Dec '99

09 Dec '99

Hello I'm pressed to tell my wishes for Christmas. Do you have suggestions of CD's about botany, zoology, or ecology ? Thanks JMV

1 0

(GEN) Lexicons
by Kevin Thiele 09 Dec '99

09 Dec '99

There is a subtext running in this discussion - whether part of our scope is the creation of lexicons or standard name-spaces - that to me is causing confusion. For instance: >>>From Leigh: >Restricting states to particular options depending on the property >in question (e.g. leaf and/or wing shape) leads back to the >prior discussion on accepted standards for character description. > >Defining, and agreeing upon these standard notations/descriptions are >A FUNDAMENTAL PART (my caps) of specifying this new format, and one that isn't >solved simply by deciding to use XML (for example). Its part of the >fundamental design and modelling, and is therefore something >that should be addressed early on. But from Leigh again: >I'd say that the DELTA approach - of avoiding domain (i.e. zoology, >virology, etc) specific notations in the format has worked well. >And I think this is the level that any initial work should be pitched at. >i.e. the data format should encode taxonomic *data* - just as DELTA >does. Any domain specific schema can be layered on top of this, or >include it. Begin with capturing the relevant data just as DELTA does, >and then progress from there. So do we or don't we? Am I misinterpreting these that they seem to say opposite things? >>>From Gregor: >It is true: Morphological structures may have containment >hierarchies, but I believe that these depent strongly on the >viewpoint of the author or user. > >EXAMPLE 2: Stuff can be in-between: The inflorescence contains part >of stem, part of leaves, and all flowers. Which leaves are part of >inflorescence and thus called bracts, and which aren't is often a >matter of taste, school, country... > >Thus: there are multiple concurrent or competing hierarchies, which >may overlap. The only problem with competing hierarchies is if we are trying to standardise and resolve the conflicts. If every worker resolves for their own project what to call bracts, this is not a problem for us. >>>From Jean-Marc: >We are designing XML vocabularies for the description of biological >species. Are we? I thought we were designing a format by which such a vocabulary can be represented. For the record, all the current systems (DELTA, LucID, NEXUS etc) enforce nothing lexically, they merely enforce a particular way of representing data. Two data sets for similar groups of plants may contain entirely different characters, or the same characters worded in different ways, or the same characters resolved into states in different ways, or (occasionally) identical characters. Comparing and combining datasets automatically is thus impossible. This seems such a shame, but is it perhaps unavoidable? Thus, if we are designing vocabularies, we are going a long way beyond what's been attempted before. Personally, I think designing domain-specific vocabularies will never work, unless the domain is the individual worker or group of collaborating workers. The popularity of lexicons is the old seductive universalism again. Great idea, but... There are two problems. Firstly, there are (broadly) two types of characters used in descriptions (and keys) - lets call them comparative and diagnostic characters. Comparative characters are the fairly general ones - e.g. leaf shape, ovary position - the sorts of characters that one would aim to describe consistently for all taxa in a monograph. Diagnostic characters are special characters that are useful for separating two or more taxa (of course, sometimes fairly general characters are diagnostic, but not always). A real example of a diagnostic character (from Synaphaea: Proteaceae): Ovary with an apical ring of translucent glands......S. bifurcata Ovary without glands.................................S. oulopha Clearly, no generalised lexicon or name-space will allow for capture of such diagnostic characters. BUT, perhaps we can have a standardised representation for the generalised characters using a lexicon and then use extensibility to allow user-specific diagnostic characters? To some extent, but perhaps not...: I will (foolishly) raise a challenge here that any generalised morphological character that anyone can come up with (in the plant domain) will be entirely inadequate for capturing data for some groups. For example, the most straightforward character I can think of is Leaves present absent But, a diagnostic difference between Discaria pubescens and Discaria nitida (Rhamnaceae) is the degree to which the leaves persist - in both, leaves tend to be absent in the adult plant, but in D. pubescens they are often completely absent while in D. nitida there are usually scattered reduced leaves on younger branchlets. And in Podostemaceae and Utricularia there's no guarantee that a leaf-like part is a leaf, because the conventional differentiation of vegetative parts into leaves/stems doesn't hold. Mother Nature's a tricky old dame, and any character definition will be inadequate to catch her. But do we put up with the inadequacy for the advantages that the universality brings? - if it means we constrain our ability to capture data, then I'd say no. So, I'd like to suggest that we try to develop a standardised data representation, but put no constraints on character definitions whatsoever. Cheers - k Beware the Universe - it bites

1 0

Re: (GEN) Lexicons
by Leigh Dodds 09 Dec '99

09 Dec '99

> There is a subtext running in this discussion - whether part of > our scope is the creation of lexicons or standard name-spaces - that to me is > causing confusion. OK, I'll clarify what I meant by the following two quotes - there is a difference, but it may not be clear.. > >Defining, and agreeing upon these standard notations/descriptions are > >A FUNDAMENTAL PART (my caps) of specifying this new format, and > one that isn't > >solved simply by deciding to use XML (for example). Its part of the > >fundamental design and modelling, and is therefore something > >that should be addressed early on. Here I'm refering standard(ised) ways of recording the data that gets stored in the format - e.g. "leaf shapes will be selected according to the IAPT plain shape chart", "all dates will be in dd-mmm-yyy format". To make the data as interoperable as possible agreeing on standard techniques and measures will help. This is not related to the definition of the format itself - it just defines how its used. My main point being that any data format (new or otherwise) isn't beneficial by itself, unless you know how the data was collected. If that can rely on standard methods then fine. If not then the method of collection needs to be expressed. > >I'd say that the DELTA approach - of avoiding domain (i.e. zoology, > >virology, etc) specific notations in the format has worked well. > >And I think this is the level that any initial work should be pitched at. > >i.e. the data format should encode taxonomic *data* - just as DELTA > >does. Any domain specific schema can be layered on top of this, or > >include it. Begin with capturing the relevant data just as DELTA does, > >and then progress from there. Here I agree with the points you make below - DELTA and the other formats work because they don't attempt to enforce anything lexically. Lets pitch the effort at that level. It would be a mammoth and probably impossible task to capture all possible data usages. >>>From that starting point particular groups can then take that standard and say "when we're recording leaf shapes they will be selected according to the IAPT plain shape chart". This is a layer above a standard data format. Here we're defining its usage. This is where the interoperability problem (sharing data between groups) will be solved. As I say, it may not be possible to make bold statements like "use the IAPT plain shape chart" in every domain, but it may be possible in some domains, or in parts of some domains. This is the layer at which the interesting stuff begins, without it you've just got a file format which happens to look the same for everyone. With agreed usage guidelines, it becomes possible to share data, and make the kind of inferences and extrapolations that we're lead to believe RDF, etc will be able to give us. > So do we or don't we? Am I misinterpreting these that they seem to say > opposite things? Bottom line - I'm talking about two different layers : - a general format for data capture - (possibly domain specific) standardised usage guidelines Does that help? > The only problem with competing hierarchies is if we are trying to > standardise and resolve the conflicts. If every worker resolves for their > own project what to call bracts, this is not a problem for us. Exactly - each worker can resolve their own hiearchy. But if a hierarchy could be agreed between groups then you start to get more use from your data. However any agreement isn't likely to occur between domains (zoology, bryology, virology), but possibly within a domain. > For the record, all the current systems (DELTA, LucID, NEXUS etc) enforce > nothing lexically, they merely enforce a particular way of representing > data. Two data sets for similar groups of plants may contain entirely > different characters, or the same characters worded in different ways, or > the same characters resolved into states in different ways, or > (occasionally) identical characters. Comparing and combining datasets > automatically is thus impossible. This seems such a shame, but is > it perhaps unavoidable? Possibly - but the data format doesn't solve that. Its how you use it. > So, I'd like to suggest that we try to develop a standardised data > representation, but put no constraints on character definitions > whatsoever. We're in agreement here - I'm just suggesting that while there are no constraints codified in the standard, they still need to be expressed (if only in usage guidelines) if you want to be able to share/exchange data meaningfully. Cheers, L.

1 0

RQTS : tag order
by Robert A. (Bob) Morris 08 Dec '99

08 Dec '99

Here's a proposed requirement for data exchange, brought home by one of Leigh's examples: <RQT> It should be possible from the data itself, perhaps together with external resources to which it refers, for software to determine whether the order matters for any collection of tags in the data.</RQT> One answer is "order always matters". That would be regrettable. For example, being able to detect that some list is unordered can make it easier to know whether two data streams represent the same object. (That's probably a deeper desideratum.) I am sure this question is addressed in most existing formats; I'm only arguing it needs to be memorialized as a requirement. I emphasize "for software to determine". I don't admit the case in which "it's in the spec" is offered as the answer to this question, unless the spec is something formal, accessible, and the question can be answered algorithmically from it. (This is an argument for making specs be in the same syntax as data, but that's a solution, not a requirement. It's also an argument why all languages but Lisp are pointless, but I bet I won't get much sympathy for that position here... :-) ). BTW, Leigh, your Subj: (XML)shearing files between programs without data loss ^^^^^^^^ I wonder: <rhetorical> an intended pun or just a typo? </rhetorical> Bob Morris

1 0

Re: (XML)Vocabularies getting out of hand
by Leigh Dodds 07 Dec '99

07 Dec '99

> Getting messy - I can't imagine that there's much shared > vocabulary between, say, mammals, insects, nematodes, sponges..., just to take a few > groups from the "zoology vocabulary". And if you asked taxomomists in each of > these fields to use a common vocab they'd complain about square pegs in > round holes. I'd say that the DELTA approach - of avoiding domain (i.e. zoology, virology, etc) specific notations in the format has worked well. And I think this is the level that any initial work should be pitched at. i.e. the data format should encode taxonomic *data* - just as DELTA does. Any domain specific schema can be layered on top of this, or include it. Begin with capturing the relevant data just as DELTA does, and then progress from there. This comes back to a point I believe I made before: if the current format(s) meet all the current and future requirements then theres no need to change things. If they don't then the work needs to focus on bridging the gap between the shortfall in requirements. L.

1 0

Re: (XML)shearing files between programs without data loss
by Leigh Dodds 07 Dec '99

07 Dec '99

> Several users have reported problems of data loss when passing files > between programs. With XML and the Document Object Model, this will > never happen, because it's easy for a programmer to work on the document > tree, just adding nodes where additions have been made by user, and > leaving the other parts unchanged. So a round-trip beetwen programs will > be feasible, leading to a composite document having a common core of > elements understood by both programs, some elements understood by > program A only, and some by program B only. Indeed, so long as the subset of elements shared by the two applications are handled correctly in both cases. Application A may decide to encode a date as simply year-month-day. Application B may decide to encode it as year-month-day-time. Load file from Application B into Application A and unless you're careful you may lose precision. Also consider the following: <character number="8"> <description>outer edge of front wing</description> <comment>shape</comment> <multi type="unordered"> <state number="1">convex</state> <state number="2">straight</state> <state number="3">concave</state> <state number="4">irregular</state> </multi> </character> <application-B-tag references="8"> I referred to standard chart X to finally decide on shapes. </application-B-tag> Application A, ignoring the application B specific tag, on request of the user deletes character 8. Now load the file up into application B and you've got a problem - you've lost the character being referenced. XML does not solve these problems. Although it brings us closer to a toolkit that does. Data modelling and agreeing on standard (extension formats) will help to solve these problems. See http://www.bath.ac.uk/~ccslrd/delta/#adv, particularly "xml is not a magic bullet" and the references to postings on the XML-DEV mailing list. Sorry to pick holes, but I'd hate for XML to be oversold to the participants on this list. Its important to see beyond the hype. Cheers, L.

1 0