September 2000 - tdwg-content

Re: Draft Spec mark 2
by Eric Zurcher 04 Sep '00

04 Sep '00

At 08:13 2/09/2000 +1000, Kevin Thiele wrote: >These are four (very early) examples, not four models. My intention is to >develop a >specification that can allow data represented with varying levels of >complexity, from very simple statements to fully worked up integral data >files such as Lucid and DELTA use. I'm not sure why, but I like the idea >that the most basic document would simply say "Eric Zurcher has two feet". >Perhaps current programs wouldn't be able to make much use of such a >document, but maybe in the future we may have a way of collecting and >organising clouds of such basic scraps (atoms) of description into something >quite powerful. Well, perhaps. But I think that our successors will appreciate it if we apply a bit of rigour and take care to describe our data with as much clarity and lack of ambiguity as possible. I don't think they'd find the statement "Eric Zurcher has two feet" to be particularly interesting or even unambiguous. Is "two feet" an anatomical statement, or a measurement (e.g., has two feet of figurative rope, by which he can easily hang himself in a forum such as this), or both <G>? >.... >| I might also note that I strongly question the way the above is organized. >| I think a rearrangement better expressing the relationships (but still not >| really addressing the problem of a lack of meaningful validation) would be >| more along the lines of: >| >| <ITEM> >| <ITEM_NAME> Gouania exilis </ITEM NAME> >| <ELEMENT> >| <ELEMENT_NAME> Flower colour </ELEMENT_NAME> >| <VALUE> green >| <QUALIFIER> rarely </QUALIFIER> >| </VALUE> >| </ELEMENT> >| </ITEM> > >This is perhaps a subtle point, perhaps a trivial one, and I'm glad you >raised it. Why is this better that the above? What exactly is the advantage? >I know yours is more DELTA-like and I agree it seems more intuitively >correct, but is there more to your strong belief than that? In truth, I >played with different ways of structuring unitary statements and presented >the one I did to draw comment rather than just follow down the path of what >we do already. I think there is a real difference in terms of a reduction in ambiguity. We should generally make an effort to clearly associate modifiers with the object that they are intended to modify. Suppose we have something like the following: <ELEMENT> <ELEMENT_NAME> leaf </ELEMENT_NAME> <VALUE> lobed margins </VALUE> <VALUE> with spines </VALUE> <QUALIFIER> rarely </QUALIFER> </ELEMENT> How should such a construct be interpretted? Do spines cover the leaf as a whole, or are they confined to the lobed margins? Is it the spines which are rarely present, or the lobed margins? Eric Zurcher CSIRO Division of Entomology Canberra, Australia E-mail: ericz(a)ento.csiro.au

1 0

Re: Draft Spec mark 2
by P. Bryan Heidorn 04 Sep '00

04 Sep '00

This is a problem but an easy one (I think). In natural language adjectives, adverbs and the like have a scope defined by the grammar that we learn as children... or are born with but we will not get into THAT here. In any case the qualifier modifier modifies the value of the character, not the character as a whole. So in the standard, any value should optionally be able to be qualified by placing the qualifier marker in the scope of the value marker. If we intend the spines to be rare not <ELEMENT_NAME> leaf </ELEMENT_NAME> <VALUE> lobed margins </VALUE> <VALUE> with spines </VALUE> * <QUALIFIER> rarely </QUALIFER> </ELEMENT> but, <ELEMENT_NAME> leaf </ELEMENT_NAME> <VALUE> lobed margins </VALUE> <VALUE> with spines <QUALIFIER> rarely </QUALIFER> </VALUE> </ELEMENT> order would not matter so we could say <VALUE> <QUALIFIER> rarely </QUALIFER> with spines </VALUE> The word "with" should not be in any of this but it is another issue. At 10:29 PM 9/4/00 +1000, Jim Croft wrote: >I agree with Eric on this one... in the example below, there is nothing in >the structure to say where the qualifier belongs... If you read the data >sequentially you might have a pretty good guess, but a specification that >relies on sequence rather than the structural relationships and clear >(explicit?) definitions of components for sense and meaning, is an bit >fragile in my view... > >jim > >>I think there is a real difference in terms of a reduction in ambiguity. We >>should generally make an effort to clearly associate modifiers with the >>object that they are intended to modify. Suppose we have something like the >>following: >> >><ELEMENT> >> <ELEMENT_NAME> leaf </ELEMENT_NAME> >> <VALUE> >> lobed margins >> </VALUE> >> <VALUE> >> with spines >> </VALUE> >> <QUALIFIER> >> rarely >> </QUALIFER> >></ELEMENT> >> >>How should such a construct be interpretted? Do spines cover the leaf as a >>whole, or are they confined to the lobed margins? Is it the spines which >>are rarely present, or the lobed margins?

1 0

Re: Draft Spec mark 2
by Mike Dallwitz 04 Sep '00

04 Sep '00

> From: Kevin Thiele <kevin.thiele(a)PI.CSIRO.AU> > To: TDWG-SDD(a)USOBI.ORG > The basic unit (atom) of a description is a statement of the form > "For item x, element y has Value z" In DELTA, character values in a description can't be separated from the 'attribute' which contains them, without losing information. This is because the order of the values, and the words 'or', 'and', and 'to' which may connect them, are significant. Also, the position of a qualifier (before or after a value) may be significant, and some qualifiers, e.g. 'reliability', may apply to the attribute as a whole rather than to a particular value. Example: 5,3/3&1<when infected with virus>/<occasionally>2<@reliability 3> = Flowers red, or red and white (when infected with virus), or occasionally yellow. This doesn't seem to be accommodated in the proposal - at least, it's not spelled out. -- Mike Dallwitz CSIRO Entomology, GPO Box 1700, Canberra ACT 2601, Australia Phone: +61 2 6246 4075 Fax: +61 2 6246 4000 Email: md(a)ento.csiro.au Internet: biodiversity.uno.edu/delta/

1 0

Response to Jim
by Kevin Thiele 04 Sep '00

04 Sep '00

Jim wrote | >3. As I see it the spec itself would define a set of allowable qualifiers | >(such as "rare" (= "rarely"?), "by misinterpretation", "uncertain" etc). I | >think we could probably agree on a limited set of qualifiers, and stick to | >that (with allowance for extension). If we do this, then "anchovies, | >please!" will be out for several reasons. | It would be nice to put a cap on the number/type of 'qualifiers' allowed, | but you can bet that there will always be something on a list that people | will want/need to use, but at this stage we probaly should confine ouselve | to the need for the list itself, not to the content of the list? | | in theory yes, but in practice, if you open this can of worms, the | discussion will probably go on for weeks... I think there's a difference between the spec defining allowable qualifiers and defining allowable values for allowable characters (the old name-space or lexicon chestnut). I've argued strongly in the past against the latter, but I do think the former is tractable. | >Two notes: | >* I'd like to allow but not enforce that a valid document have the | >allowable_values property filled. By not enforcing it, a simply marked-up | >natural-language description could be a valid document. This would perhaps | >mean that the spec could meet half-way the automated markup of legacy | >descriptions, and I'm keen to do this. Of course, a document without this | >property specified would not be able to be validated in the way you suggest | >and hence may not be very useful for some purposes, but this may be a price | >one is willing to pay for other benefits, and I think we need to keep this | >open. | | It would be pretty difficult to validate a blob of free text... but I can | see a case for wanting to include them in the exercise... so perhaps there | should be a deprecated class of elements that are essentially unable to be | validated but perhaps of interest in some instances? We need to remember that 99.99999% of all the descriptive data in the world has not been validated except by error-prone humans. I think the bulk of the endeavour of the last few centuries is still a pretty valuable resource. | >* I'm using "allowable values" rather than "states" as this seems to me to | >be more general, and subsumes the requirement to distinguish between, for | >instance, multistate and numeric "characters". A numeric "character" of | >course, doesn't have "states", but it does have allowable values (integers | >in the case of an integer numeric, real numbers in the case of a real | >numeric). | | This would cover the use of text rather than various numeric values - there | is nothing that should say everything as to be represented numerically. It | should be possible to use text *and* to have it validated in some way. Yes, but I don't mean using a character of type TEXT. I don't think this is necessary. | >1. How strong is the requirement for this type of validation? Enforcing this | >seems to me to be like requiring that all word documents carry in their | >header a dictionary of the english language to allow validation of | >spellings. It seems to me that providing tools that allow people to check | >these strings against a predefined list (defined either within the document | >or in an external resource) would be useful, but not absolutely necessary. A | >document that is not or cannot be validated in this way would not be | >useless, and would perhaps be more free. | | Documents do not have to carry their validation with them - they could | refer to an external source of validation (or dictionary in your above | example - this is what is implied and expected in Word documents at the moment) Exactly. A document that is validated against an external resource is more valuable than one not validated. That makes the validated document more valuable, but it doesn't make the non-validated one value-less (if that were the case Jim all your emails, with their liberal sprinkling of typos, would be worthless!) | >Note that the spec as I see it would allow (but again, not enforce as DELTA | >and Lucid do) the encoding of descriptions. Thus, a valid document may be d1 | >as below. This would preempt the need for typographic validation, and allow | >allowable-values validation. But for some reason I don't want to disallow d2 | >as a valid document also. | | This may be a non threatening approach to gradually introducing validity | and rigour into descriptive data and is probably worth exploring some more... A non-threatening approach ... gun for it! Cheers - k

1 0

Re: Validation
by Jim Croft 04 Sep '00

04 Sep '00

Bryan wrote: >We can validate existence if not content. I should be able to say that all >of my >descriptions will have <natural history> notes and maybe <medical uses> >in regular old natural language. If I set up my project standard correctly >under SDD >my validator should scream is an author tries to send in a manuscript >without that >section, even if the computer can not read the text. I think this is a point that Kevin did not pick up clearly in his initial pass at validation and should really be part of the specs... >I would like to see a "Free Text" character type in the standard and warn >authors >that they can not use it for characters that they want to process through >interactive key programs. That is more or less what happens in DELTA at the moment, isn't it? jim

1 0

What I'm trying to do
by Kevin Thiele 04 Sep '00

04 Sep '00

Jims and Bryan's comments are exactly where we want to be at the moment, methinks. I've realised over the weekend, thinking about Eric Zurcher's criticisms, what it is I'm trying to do with Draft Spec Mark 2 (some people are a bit slow). In Lucid and DELTA we enforce a great deal of structure on the data file: *CHARACTER LIST #1. Distribution by State (or known provenance)/ 1. South Australia - south-eastern (south of the line from Port Augusta to Broken Hill)/ 2. New South Wales (including Jervis Bay - A.C.T.)/ 3. Australian Capital Territory/ 4. Victoria/ 5. Tasmania/ ...etc If the * or the # or a / is left off or put in the wrong place, the whole thing falls over. Only one very narrowly constrained data format can be a valid file. This is primarily done for ease of processing, and was a perfectly reasonable constraint for DELTA and Lucid to enforce, since what they were trying to do was create a data file for their particular program. The standard I'm working towards ALLOWS this degree of formality, but doesn't ENFORCE it. In the standard, for a file to be valid input for Lucid or DELTA, it would need to conform to higher-order structure as imposed by those programs. But not all descriptive data out there is in DELTA or Lucid format, as Bryan notes, and we need to be inclusive of other types of data as well (by far the majority of which is natural-language legacy data). If all descriptive data needs to be encoded to a strong specification, it will never be so encoded (that's one reason, I think, why DELTA has failed as a global specification). So I'm trying to create a spec where both this: <DOCUMENT Name = "d1"> <ITEM_PROPERTIES> <ITEM ID = "1" NAME = "Gouania exilis"/> <ITEM ID = "2" NAME = "Gouania australiana"/> </ITEM_PROPERTIES> <ELEMENT_PROPERTIES> <ELEMENT ID = "1" > <ELEMENT_NAME> Flower colour <\ELEMENT NAME> <VALUE_LIST> <VALUE ID = "1"> <VALUE_NAME> "green" </VALUE_NAME> </VALUE> <VALUE ID = "2"> <VALUE_NAME> "yellow" </VALUE_NAME> </VALUE> </VALUE_LIST> </ELEMENT> </ELEMENT_PROPERTIES> <DESCRIPTION Name = "Gouania exilis"> <ELEMENT> <ELEMENT_ID> 1 </ELEMENT_ID> </ELEMENT> <VALUE_ID> 1 </VALUE_ID> <QUALIFIER> rarely </QUALIFIER> </DESCRIPTION> </DOCUMENT> ... and this ... <DOCUMENT Name = "d2"> Viola eminens K. Thiele & Prober, sp. nov. <DESCRIPTION Name = "Viola eminens"> <ELEMENT = "Longevity"> <VALUE>Perennial</VALUE> </ELEMENT> <ELEMENT = "Life form"> <VALUE> herb </VALUE> </ELEMENT> spreading by stolons; rootstock sometimes somewhat swollen and bulbous at the stem bases. Stems contracted so that the leaves form rosettes, never elongate with caulescent leaves. <ELEMENT Name = "Leaves">Leaves <ELEMENT Name = "lamina"><ELEMENT Name = "Shape"><VALUE>broad-reniform</VALUE></ELEMENT>, the largest (10-)12-15(-25) mm long, (20-)25-35(-45) mm wide, 1.5-3.2 times wider than long, usually with a broad basal sinus; lamina with 9-20 +/- prominent teeth, glabrous or with scattered unicellular hairs on the upper surface, +/- concolorous bright green </ELEMENT>; petioles 2-8 cm long; stipules narrowly triangular, usually with several small, glandular teeth on each side.</ELEMENT> .........etc </DESCRIPTION> </DOCUMENT> are valid documents. Now there's a lot of blob text in d2, but it's still a description, and surely the very simple markup has value-added enormously to it. If we find this document through a web search we at least know that it includes a description of Viola eminens, and that this description includes a statement about the life form of that species. This is a huge advance on only knowing that we've found a document that contains the words "Viola" and "eminens". Isn't that what XML is all about? Further, it seems it me that imposing the most basic formalities only (that a description is about something, that it starts at <DESCRIPTION> and ends with </DESCRIPTION>, and that it may optionally include some structured statements of the form <ELEMENT></ELEMENT> etc) can actually surprisingly easily allow parts at least of d2 to be mapped (or reformatted) to d1 (once we've got the syntax right which isn't yet the case with d1 and d2, he hastens to add). It seems to me that creating a standard like this would be more valuable than simply XMLifying the DELTA of Lucid data file structure. So that's what I'm trying to do, I think. Cheers - k

1 0

Re: Philosophy; Was: Re: Draft Spec mark 2
by Jim Croft 04 Sep '00

04 Sep '00

Bob Wrote: >Kevin's effort, and TDWG's charter, is foremost about data in the >service of knowledge, rather than knowledge in the service of >users. As such, his effort supports the system builders, especially >those with a desire to exchange or federate data or (one hopes) to >build extensible systems. I like this first sentence especially as it sums up the level that TDWG is, or should be on about... >Taxonomists, God bless 'em. You can't live with 'em and you can't >live without 'em. Well, biological information management would certainly be a lot easier if they hadn't got there first... :) jim

1 0

Re: Draft Spec mark 2
by Jim Croft 04 Sep '00

04 Sep '00

Una wrote: >But the discussion here keeps slipping into implementation issues >and forgetting about the underlying abstractions. This is so true and has dogged this list ever since it started - at this stage we need to be constantly on guard against dropping to quickly into building something before we have properly defined the scope of the project. >We could conduct this discussion in PERL code too. (Hey, why not?) Because PERL is yucky, that's why... :) >Why not forget the codes >(or pseudo codes) and use ordinary words that we all understand? *Then* >decide which language (if any) to implement it in, *after* we all agree >what "it" is. That is what I am waiting for... XML seems to be a fairly intuitive contemporary way of describing things and it seems to be a reasonable thing to use for the time being... Remember our very first attempts to do it in EBNF? Same idea, different language, but at the time, I do not think many of us were aware of what we were trying to achieve... This was a graphic example of an idea probably being right, but if people can not understand and embrace it, it will not get anywhere... Still not 100% sure what 'it' is, but I think with the most recent discussions we are getting closer... jim

1 0

Re: Validation
by Jim Croft 03 Sep '00

03 Sep '00

Kevin wrote: >I'm not sure about a requirement for strong validation in the spec. I agree >that validation should be allowed, but not required. But isn't that the main (only?) reason for bothering to encode descriptive data in the first place? To try and impose a degree of consistency, rigour, comparability and validity that is not possible in an unstructured, unformatted blob of text? If the specifications are not going to require this degree of strandardization, are we going to be letting the tansferability of data and effort that we seek slip by? The value of the DELTA, Lucid and other character-based encoding of descriptive data is that they do impose a high degree of validation, whether it be by measurement ranges, allowable values or states, or whatever. Without validation in biological datasets you end up with a mess - just look at any herbarium or museum specimen dataset - who doesn't have terrestrial specimens that plot in the middle of the ocean? >There are three types of validation you suggest here: >1. Validate the element strings (and presumably also items, values and >qualifiers) against predefined lists to guard against typographical and >spelling errors; and for consistency... one can be right, but inconsistent, and although humans can handle this, computers stuff it up... >2. Validate values against a list of allowable values for the character; This is the mechanism to addressing the above... >3. Validate qualifiers against a list of allowable qualifiers. It would be nice to put a cap on the number/type of 'qualifiers' allowed, but you can bet that there will always be something on a list that people will want/need to use, but at this stage we probaly should confine ouselve to the need for the list itself, not to the content of the list? >Taking these in reverse order: as one would naturally want to do.. :) >3. As I see it the spec itself would define a set of allowable qualifiers >(such as "rare" (= "rarely"?), "by misinterpretation", "uncertain" etc). I >think we could probably agree on a limited set of qualifiers, and stick to >that (with allowance for extension). If we do this, then "anchovies, >please!" will be out for several reasons. in theory yes, but in practice, if you open this can of worms, the discussion will probably go on for weeks... >2. Validation of allowable values is covered in the draft spec. One of the >properties of an element (~"character") is a list of allowable values >(~"states"). If such a property is filled, then validation can be done by >the program that handles the data, just as in DELTA and Lucid. It is not just 'allowed', isn't it? Isn't this what the draft specs are all about? >Two notes: >* I'd like to allow but not enforce that a valid document have the >allowable_values property filled. By not enforcing it, a simply marked-up >natural-language description could be a valid document. This would perhaps >mean that the spec could meet half-way the automated markup of legacy >descriptions, and I'm keen to do this. Of course, a document without this >property specified would not be able to be validated in the way you suggest >and hence may not be very useful for some purposes, but this may be a price >one is willing to pay for other benefits, and I think we need to keep this >open. It would be pretty difficult to validate a blob of free text... but I can see a case for wanting to include them in the exercise... so perhaps there should be a deprecated class of elements that are essentially unable to be validated but perhaps of interest in some instances? >* I'm using "allowable values" rather than "states" as this seems to me to >be more general, and subsumes the requirement to distinguish between, for >instance, multistate and numeric "characters". A numeric "character" of >course, doesn't have "states", but it does have allowable values (integers >in the case of an integer numeric, real numbers in the case of a real >numeric). This would cover the use of text rather than various numeric values - there is nothing that should say everything as to be represented numerically. It should be possible to use text *and* to have it validated in some way. >1. How strong is the requirement for this type of validation? Enforcing this >seems to me to be like requiring that all word documents carry in their >header a dictionary of the english language to allow validation of >spellings. It seems to me that providing tools that allow people to check >these strings against a predefined list (defined either within the document >or in an external resource) would be useful, but not absolutely necessary. A >document that is not or cannot be validated in this way would not be >useless, and would perhaps be more free. Documents do not have to carry their validation with them - they could refer to an external source of validation (or dictionary in your above example - this is what is implied and expected in Word documents at the moment) >Note that the spec as I see it would allow (but again, not enforce as DELTA >and Lucid do) the encoding of descriptions. Thus, a valid document may be d1 >as below. This would preempt the need for typographic validation, and allow >allowable-values validation. But for some reason I don't want to disallow d2 >as a valid document also. This may be a non threatening approach to gradually introducing validity and rigour into descriptive data and is probably worth exploring some more... jim

1 0

Re: Validation
by P. Bryan Heidorn 02 Sep '00

02 Sep '00

At 08:19 AM 9/3/00 +1000, Jim Croft wrote: >Kevin wrote: >>I'm not sure about a requirement for strong validation in the spec. I agree >>that validation should be allowed, but not required. > >But isn't that the main (only?) reason for bothering to encode descriptive >data in the first place? To try and impose a degree of consistency, >rigour, comparability and validity that is not possible in an >unstructured, unformatted blob of text? If the specifications are not >going to require this degree of strandardization, are we going to be >letting the tansferability of data and effort that we seek slip by? > >The value of the DELTA, Lucid and other character-based encoding of >descriptive data is that they do impose a high degree of validation, >whether it be by measurement ranges, allowable values or states, or >whatever. Without validation in biological datasets you end up with a mess >- just look at any herbarium or museum specimen dataset - who doesn't have >terrestrial specimens that plot in the middle of the ocean? I agree that there should be the capability to do strong validation but as Jim stated below there are times when "free" text is appropriate in a taxonomic description. I'll go farther than Jim however and say free text fields are not only interesting for discussion but necessary elements of the standard. If we do not allow at least some free text in the descriptions then the standard will not be applicable to many worthy endeavors where it would be of great use. The matter hinges on the type of taxonomic descriptions that we want the standard to cover. We should discuss some of them. Since there was a little confusion about what is meant by text I'll distinguish between two types. Controlled Text comes from a controlled vocabulary. It is proscribed like the controlled vocabularies used in libraries for indexing. "Free text" is just words in the wild. It can be anything that the writer intends including anchovies. For example, Kevin has argued that the qualifiers should come from a controlled and therefor validatable vocabulary. Two Types of Taxonomic Description: (not even a little exhaustive but certainly exhausting) Certainly the first type of taxonomic description is one that is used by computer identification systems like IntKey and Lucid. For this type of taxonomic description we need strong type checking and validation. It would need controlled vocabularies, no anchovies for qualifiers, numeric checking, range checking... The second type of description is an entry from a field guide to butterflies of the arctic. It is mostly intended for direct human consumption without much prior chewing by a computer. Even in this case there might be good reason to allow the field guide entry to follow the standard. For example, assuming that the field guide is on-line we might want to do some name validation and indexing in the Global Taxonomic Workbench. Also, even field guides have keys sometimes. We shouldn't just ignore interesting text like "The wood of this tree makes wonderful canoes." I do not mean that we should have a <Canoe-Quality> character. We'll never be able to capture everything in characters. There is certainly a substantial middle ground in types. For example the first document type, the key matrix data, might be run through a program to produce at least acceptable natural language descriptions. This has proven useful for some projects using Delta. Like Delta we'll need to allow "holes" in the validation so that we can include text to make things a little more readable for us humans. > >>3. Validate qualifiers against a list of allowable qualifiers. > >It would be nice to put a cap on the number/type of 'qualifiers' allowed, >but you can bet that there will always be something on a list that people >will want/need to use, but at this stage we probaly should confine ouselve >to the need for the list itself, not to the content of the list? Perhaps the standard could have a suggested set or two but require that is qualifiers are used that they be from a project managed controlled vocabulary. This is a little like Z39.50 allowing different attribute sets but still being a standard. I did say a _Little_ like Z39.50. > >>Taking these in reverse order: > >as one would naturally want to do.. :) > >>3. As I see it the spec itself would define a set of allowable qualifiers >>(such as "rare" (= "rarely"?), "by misinterpretation", "uncertain" etc). I >>think we could probably agree on a limited set of qualifiers, and stick to >>that (with allowance for extension). If we do this, then "anchovies, >>please!" will be out for several reasons. > >in theory yes, but in practice, if you open this can of worms, the >discussion will probably go on for weeks... Oh, I am that we'll find some other trivia to argue about for weeks if not this. > >>2. Validation of allowable values is covered in the draft spec. One of the >>properties of an element (~"character") is a list of allowable values >>(~"states"). If such a property is filled, then validation can be done by >>the program that handles the data, just as in DELTA and Lucid. > >It is not just 'allowed', isn't it? Isn't this what the draft specs are >all about? Usually, but we do need to make it clear in documents following the standards which pieces are validatable and how they are validated. > >>Two notes: >>* I'd like to allow but not enforce that a valid document have the >>allowable_values property filled. By not enforcing it, a simply marked-up >>natural-language description could be a valid document. This would perhaps >>mean that the spec could meet half-way the automated markup of legacy >>descriptions, and I'm keen to do this. Of course, a document without this >>property specified would not be able to be validated in the way you suggest >>and hence may not be very useful for some purposes, but this may be a price >>one is willing to pay for other benefits, and I think we need to keep this >>open. > >It would be pretty difficult to validate a blob of free text... but I can >see a case for wanting to include them in the exercise... so perhaps there >should be a deprecated class of elements that are essentially unable to be >validated but perhaps of interest in some instances? We can validate existence if not content. I should be able to say that all of my descriptions will have <natural history> notes and maybe <medical uses> in regular old natural language. If I set up my project standard correctly under SDD my validator should scream is an author tries to send in a manuscript without that section, even if the computer can not read the text. Also computer programs processing the text should be able to tell what section they are in. For example people should be able to search just <Medical uses> section for the word "eye". If you can not do that you might end up with a list of all creatures with eyes and not medicines for your own eye. Finally, there is free text in legacy data. It is reasonable to write computer programs that process this data and "mark-up" interesting things at least weakly following the standard. We might be able to have programs find the Latin name, common names, authority, bibliography and other useful things and tag them for future use.It would be impossible however to make programs that convey the entire contents of these free text documents into the highly structured "interactive key" quality data sets. > >>* I'm using "allowable values" rather than "states" as this seems to me to >>be more general, and subsumes the requirement to distinguish between, for >>instance, multistate and numeric "characters". A numeric "character" of >>course, doesn't have "states", but it does have allowable values (integers >>in the case of an integer numeric, real numbers in the case of a real >>numeric). I would like to see a "Free Text" character type in the standard and warn authors that they can not use it for characters that they want to process through interactive key programs. > >This would cover the use of text rather than various numeric values - there >is nothing that should say everything as to be represented numerically. It >should be possible to use text *and* to have it validated in some way. > Right >>1. How strong is the requirement for this type of validation? Enforcing this >>seems to me to be like requiring that all word documents carry in their >>header a dictionary of the english language to allow validation of >>spellings. It seems to me that providing tools that allow people to check >>these strings against a predefined list (defined either within the document >>or in an external resource) would be useful, but not absolutely necessary. A >>document that is not or cannot be validated in this way would not be >>useless, and would perhaps be more free. > >Documents do not have to carry their validation with them - they could >refer to an external source of validation (or dictionary in your above >example - this is what is implied and expected in Word documents at the moment) Right, systems work this way now. We could store the standard the a description is supposed to follow anywhere in the world. At the beginning of the document place text saying where to find it. <SDD Version 1.10 http://global.taxonomy.org/SDD1.1/ Gouania > Assuming some group had established the character lists for Gouania and registered them with global.taxonomy.org. Like W3 stores standards. > >>Note that the spec as I see it would allow (but again, not enforce as DELTA >>and Lucid do) the encoding of descriptions. Thus, a valid document may be d1 >>as below. This would preempt the need for typographic validation, and allow >>allowable-values validation. But for some reason I don't want to disallow d2 >>as a valid document also. > >This may be a non threatening approach to gradually introducing validity >and rigour into descriptive data and is probably worth exploring some more... Bravo > >jim Bryan

1 0