February 2000 - tdwg-content

Re: SDD Specifications Document
by Kevin Thiele 29 Feb '00

29 Feb '00

At 10:13 25/02/00 -0300, Mauro wrote: >I was about to suggest that a graphic model, perhaps using the UML >methodology, was in order, but Jean-Marc Vanel has already taken care of >that. His model is interesting, and deserves attention. I agree that a graphical representation of the model may be a good idea, but it's beyond me. Jean-Marc's work does indeed need appraisal by the group in relation to our goals. My understanding from a brief perusal of his site is that he's trying to establish a franework for creating a set (one day he hopes for a global set) of descriptions of plants on the web, the data structured in such a way as to be queryable as well as readable. So, has Jean-Marc already done all that this group set out to do and can we go home? Two likely differences between the present attempt and his may be 1. that we need to create something more flexible so that any botanist can capture any aspect of systematic data for any taxon - i.e. a Stipa can be described just as fully as a Taraxacum and a Nepenthes, with data ranging from morphology to anatomy, phytochemisty and gene sequences (the latter is the only easy one). Jean-Marc's thing seems to me to be of the fairly inflexible variety (see the earlier lexicon debate) i.e. systematic description is reduced to filling in the blank spaces on the form provided: Taxon name <fill in your taxon> Leaves <phyllotaxy>; margins <indentation>; venation <type>. I may be misrepresenting Jean-Marc's intentions here. Any comment? 2. Jean-Marc's model is directed towards capturing descriptions. We need to also allow for identification data for the range of key programs available and in the future. Thus, things like character sets, dependencies, allowing for misinterpretations etc may perhaps have little meaning in Jean-Marc's domain, but are critical in ours. Does anyone think we shold discontinue our attempt and run with Jean-Marc's instead? At the very least, we should make sure that they're compatible, perhaps seeing Jean-Marc's as a special-purpose subset of the general. -k

1 0

XML Schema
by Robert A. (Bob) Morris 28 Feb '00

28 Feb '00

Those who care may already know that on Feb 25 W3C released a working draft for public comment of XML-Schema, with a last call for review expected in March. Perhaps of more general interest is XML Schema Part 0: Primer at http://www.w3.org/TR/xmlschema-0/ >>>From the abstract: "XML Schema Part 0: Primer is a non-normative document intended to provide an easily readable description of the XML Schema facilities and is oriented towards quickly understanding how to create schemas using the XML Schema language."

1 0

Re: Progressive Revelation
by Mike Dallwitz 28 Feb '00

28 Feb '00

> From: Kevin Thiele <kevin.thiele(a)PI.CSIRO.AU> > To: TDWG - Structure of Descriptive Data <TDWG-SDD(a)USOBI.ORG> > Create a key to all grass species so you're working with a list of all > taxa at species level including all the Poa species. The character list > has two classes of characters - ones that are scored over all taxa (these > will be the easily generalised characters) and ones that are scored for > only a subset of taxa (the characters that are highly specific and/or not > easily generalisable). When the key program starts it splashes up the > generalised characters only. But if after answering some characters you > end up with only Poas, the program finds and adds to the character list > the Poa-specific characters. Characters are progressively revealed as you > proceed through the key, with as much depth as necessary - e.g. you may > come down to a species complex of alpine Poas and presto! some characters > appear that are just the ticket to separate them. Something like this, but less rigid, is achieved automatically by algorithms for finding 'best' characters. A 'best' algorithm typically has a penalty for characters which are unknown, inapplicable, or variable for some taxa, but it does not completely exclude them. The 'best' algorithm used in our programs Intkey (interactive identification) and Key (generation of conventional keys) has a natural penalty for such characters, arising from the goal of minimizing the average length of an identification. The algorithm also has a parameter, Varywt, which can be used to add an arbitrary penalty for such characters. However, this tends to increase the average length of an identification, so its use in Intkey is not recommended. A value of 0 for Varywt would have an effect similar to that proposed above by Kevin. If you try this in Intkey, note that a value of 0 is treated as 0.01, an ad hoc adjustment made specifically to _avoid_ the complete exclusion of the characters. In a data set with a substantial number of missing or inapplicable values (e.g. the sample data supplied with the DELTA programs), it is easy to observe how low values of Varywt cause characters with comparatively low separating power (and which therefore result in longer identifications) to move towards the top of the 'best' list. Varywt is primarily intended for use in Key, where the increased average length of an identification is offset by a reduction in the _printed_ length of the key. In Key, a value of 0 completely prevents the use of characters with intra-taxon variability. -- Mike Dallwitz CSIRO Entomology, GPO Box 1700, Canberra ACT 2601, Australia Phone: +61 2 6246 4075 Fax: +61 2 6246 4000 Email: md(a)ento.csiro.au Internet: biodiversity.uno.edu/delta/

1 0

Progressive Revelation
by Robert A. (Bob) Morris 28 Feb '00

28 Feb '00

Since theology has arisen, how's this: Object oriented databases are better for this stuff than either hierarchical or relational, precisely for the reasons you outline below. [By the way, hoping this is not a forbidden commercialism but at least reveals my conflict of interest: we operate the university program for eXcelon, Inc. (formerly Object Design) by which universities can get Object Store, eXcelon [a native XML store], and most other eXcelon products for a total of $650/year. See our web site at www.cs.umb.edu/~serl/odiedu. This is a pretty good way to get into OODB's if you are a university. The program applies in most of the world.] Bob Morris Kevin Thiele writes: > Date: Mon, 28 Feb 2000 09:47:10 +1100 > From: Kevin Thiele <kevin.thiele(a)PI.CSIRO.AU> > To: TDWG-SDD(a)usobi.org > Subject: Progressive Revelation > > At 15:30 24/02/00 +1100, Eric Zurcher wrote: > > >6) I'm intrigued by the notion of a "Progressive Revelation model" > >(footnote 5). It sounds terribly theological - or perhaps that's > >Thiele-logical? (my apologies to Kevin, but I really can't resist bad puns). > > I'm often accused of teleology, but rarely of theology. > > Progressive Revelation is perhaps a new way of handling holes in data > matrices for random-access keys. The background is this: > > The simplest data structure for a random-access key is a fully populated > matrix i.e. all taxa are scored for all characters/states. Works well > sometimes, especially if the taxa are highly comparable e.g. the species of > a genus or the genera of a family. > > This structure is problematic sometimes though, for two reasons. Firstly and > most simply, you may not have data for all taxa, and need to leave holes in > the matrix. Solution is simple - fill the holes with ?s and allow for this > in the key program. But it often also happens that some characters are > simply inapplicable to some taxa, or (worse) are non-ambiguous for some taxa > but ambiguous for others. For instance, stipules don't occur in monocots, > stipule-like structures sometimes do but if you try scoring stipule > characters as defined for dicots against monocots you run into all sorts of > strife because of ambiguity of context. LucID can handle this to some extent > using the "present by misinterpretation" score, but the problem is in the > character definition, not the score. > >...

1 0

Re: <XML> Abstract Data Model for Taxonomy
by Kevin Thiele 28 Feb '00

28 Feb '00

>>>From Jean-Marc >Leigh Dodds a écrit : > >> 1. Firstly is it possible to express items like >> "Feature often present by mis-identification" > >Sorry, my english is too weak. Does this mean a character value that is >often reported by confusion with another taxon ? Leigh is presumably referring to the "Feature present (by misinterpretation)" score (see LucID). This is a score applicable to a state for a taxon so that, for instance, if a taxon lacks petals but has petal-like pseudopetals, you can score that petals are absent (the truth) but also present (by misinterpretation). This pre-empts likely user errors in an interactive key, while allowing the treatment builder to maintain data integrity. If there is no such score, the only way of pre-empting the mistake is to deliberately introduce errors into the data.

1 0

Progressive Revelation
by Kevin Thiele 28 Feb '00

28 Feb '00

At 15:30 24/02/00 +1100, Eric Zurcher wrote: >6) I'm intrigued by the notion of a "Progressive Revelation model" >(footnote 5). It sounds terribly theological - or perhaps that's >Thiele-logical? (my apologies to Kevin, but I really can't resist bad puns). I'm often accused of teleology, but rarely of theology. Progressive Revelation is perhaps a new way of handling holes in data matrices for random-access keys. The background is this: The simplest data structure for a random-access key is a fully populated matrix i.e. all taxa are scored for all characters/states. Works well sometimes, especially if the taxa are highly comparable e.g. the species of a genus or the genera of a family. This structure is problematic sometimes though, for two reasons. Firstly and most simply, you may not have data for all taxa, and need to leave holes in the matrix. Solution is simple - fill the holes with ?s and allow for this in the key program. But it often also happens that some characters are simply inapplicable to some taxa, or (worse) are non-ambiguous for some taxa but ambiguous for others. For instance, stipules don't occur in monocots, stipule-like structures sometimes do but if you try scoring stipule characters as defined for dicots against monocots you run into all sorts of strife because of ambiguity of context. LucID can handle this to some extent using the "present by misinterpretation" score, but the problem is in the character definition, not the score. Sometimes a better way to handle such circumstances using LucID is to break the key up into a hierarchically nested set of keys & subkeys. For instance, you want to create a key to grass species of Australia but there are many special characters needed for identifying Poa species that are either inapplicable to or ambiguous with respect to the remaining grasses. So put Poa as a genus in the top-level key and attach to it a subkey to Poa species in which you can optimise your character definitions for the Poas. There are some disadvantages to this but often the advantages (in having an optimised rather than generalised and suboptimal character list) outweigh the disdvantages. But there may be another solution - Progressive Revelation. As far as I know no-one's done this yet, but I think it has merit. It would work like this. Create a key to all grass species so you're working with a list of all taxa at species level including all the Poa species. The character list has two classes of characters - ones that are scored over all taxa (these will be the easily generalised characters) and ones that are scored for only a subset of taxa (the characters that are highly specific and/or not easily generalisable). When the key program starts it splashes up the generalised characters only. But if after answering some characters you end up with only Poas, the program finds and adds to the character list the Poa-specific characters. Characters are progressively revealed as you proceed through the key, with as much depth as necessary - e.g. you may come down to a species complex of alpine Poas and presto! some characters appear that are just the ticket to separate them. Might work. This seems to me to be a more natural way of approaching both the building and running of a key. In some ways it's like a hybrid between a traditional random-access key and a traditional nested hard-copy key, but it has more flexibility than either. In the context of natural-language descriptions (and more controversially) it would also provide a challenge to what to me is the Universalist furphy that all descriptions should be strictly comparable! I've included these ideas into the SDD Specification because I think this may be one for the future. Cheers - k ps Eric - if you incorporate these ideas into DELTA we'll need to make an arrangement regarding due acknowledgement and royalties :-)

1 0

Re: SDD Specifications Document
by Kevin Thiele 28 Feb '00

28 Feb '00

At 15:30 24/02/00 +1100, Eric Zurcher wrote: >1) One "pattern" that recurs in the document in the use of "attachments" to >entities. These consist of a name, type, path, public notes, and private >notes. Presumably one could define some sort of generic "attachment" object >and avoid the multiple (4) redefinitions currently given in the outline. >This would shorten the outline appreciably. This sounds sensible. Should someone edit the document to reflect this, or should we just leave efficiency dividends like this to Leigh or whoever creates the final thing? Two comments though. Firstly, most of these attachment objects will be unique - presumably it won't often happen that one attachment will be used for two taxa, or for a taxon and a character state. So do you gain much? Secondly, ease of reading the text treatment should be a consideration. If all the information re the attachment stays with the taxon/state etc entry, this may make the treatment more navigable. >2) ID numbers crop up in a number of places. But it's not obvious how >"unique" these various IDs need to be. For example, must character IDs be >unique only with the context of the set of all character IDs, or of all IDs >used anywhere within the treatment? Or perhaps they are intended be used >across treatments (to facilitate merging, etc.), and must be unique (but >consistent) in an even broader sense? (But perhaps this is an area this is >best left deliberately ambiguous for now.) I'd say that IDs need to be unique within their context eg at the character level. Super-uniqueness beyond the treatment is the lexicon issue and doesn't need to be addressed here, I think. We should construct the standard in such a way that lexica can grow, but without a requirement for them. For instance, there should be no requirement that character IDs form a contiguous integer set. >3) Character (and taxon) sets - these should probably be defined >hierarchically, so that sets would be able contain other sets, as well as >the base elements (characters or taxa). Note that there seems to be another >"up vs. down" problem here - is it better to define a set by listing the >members within it, or for the each of the members to list the sets to which >it belongs? What do people think? Does the idea of allowing nested sets overlap and merge the boundary with the nested character structure, and perhaps become redundant. For instance, I envisage the characters being nested (as discussed earlier in the list): Plant leaves orientation venation prominence reticulation flowers petals colour number .........etc Sets are perhaps a shortcut across all this. If they were heirarchically structured, would you just be repeating much of the above without the lowest level? >4) I'm rather confused by footnote 3, regarding the nesting of character >names, and the restriction of "properties" to only the lowest level. What >it the reason for this restriction? But certainly there seems to be merit >in separating the "properties" of a character from it's textual >representation. This is almost essential when attempting to generate >natural-language descriptions in multiple languages. Similarly, different >wordings may be appropriate in different application contexts >(natural-language vs. interactive keys vs. conventional keys, or keys of >the layman vs. the specialist). There are many things in the document that I haven't thought through properly, but I decided to get it out for comment before labouring further. I think what I meant was that the higher-level structures in the character names list do not have any representation in the data "matrix", only the lowest level. Consider some data represented in the LucID way (as a taxon-state matrix) (you can do the same thing for a DELTA representation in which cells of the matrix hold taxon-character scores): 123456789 Taxon1 010101000 Taxon2 111010101 Taxon3 000101001 Columns 1-9 represent character states. Columns 1-3 may be the states of character 1, 4-5 the states of character 2, 6-9 the states of character 3. Now characters 1&2 may both belong to a higher-level structure, but the properties of this higher-level thing are not equivalent to the properties of the state (for instance, it can't have a score). But perhaps there are properties in common? >5) This draft allows for a "score" only within the context of a "state >name". It is not obvious how characters with non-discrete values (e.g. >numeric values) would be handled. Just as for Leigh's original XDELTA. >7) For purposes of natural-language generation (and perhaps other >applications), it is desirable to have some sort of "connection operator" >between states within a character (e.g., "flowers blue or violet" vs. >"flowers blue and violet" vs. "flowers blue to violet" all carry slightly >different meanings). This and other requirements of generating >natural-language descriptions might be an argument for generally preferring >a "characters within taxa" representation to "taxa within characters". Again, I think Leigh's XDELTA covers this, using nested scores.

1 0

Re: SDD Specifications Document
by Kevin Thiele 28 Feb '00

28 Feb '00

Bob Morris surely something like this - working out data requirements - needs to be done whether the final product has a DTD, RDF or XML-Schema base. I'm not qualified to argue the implications or advantages, so how about those of us like me argue about the data, and you mob argue about the rest. Maybe we can get to resolutions at the same time! Cheers - k

1 0

Re: SDD Specifications Document
by Neil Caithness 25 Feb '00

25 Feb '00

> From: "Mauro J. Cavalcanti" <maurobio(a)ACD.UFRJ.BR> > > I was also curious about this notion. Maybe it is one of the latest > "revelations" of Religious Cladism - its practitioners are fond of such > theological terminology. Mauro, are you serious, or was this just a through-away comment? This is the sort of pseudo-criticism that comes from people who understand nothing. "Religious Cladism" means what, and "its practitioners" refers to whom? Neil Caithness

1 0

Re: SDD Specifications Document
by Mauro J. Cavalcanti 25 Feb '00

25 Feb '00

Eric Zurcher escreveu: > 6) I'm intrigued by the notion of a "Progressive Revelation model" > (footnote 5). It sounds terribly theological - or perhaps that's > Thiele-logical? (my apologies to Kevin, but I really can't resist bad puns). I was also curious about this notion. Maybe it is one of the latest "revelations" of Religious Cladism - its practitioners are fond of such theological terminology. Otherwise, I essentially agree with all the points raised by Eric Z. I was about to suggest that a graphic model, perhaps using the UML methodology, was in order, but Jean-Marc Vanel has already taken care of that. His model is interesting, and deserves attention. Cheers, -- + - - - - - - - - - - - - Mauro J. Cavalcanti - - - - - - - - - - - - + | Setor de Paleovertebrados, Departamento de Geologia e Paleontologia | | Museu Nacional do Rio de Janeiro | | Quinta da Boa Vista, 20940-040, Rio de Janeiro, RJ, BRASIL | | E-mail: maurobio(a)acd.ufrj.br | | Home Page: http://read.at/digitax/personal.html | + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + "Life is complex. It consists of real and imaginary parts."

1 0