December 2001 - tdwg-content

Re: It's How the Data will be Used that Counts
by unknown＠example.com 04 Dec '01

04 Dec '01

Perfect agreement with Bob? Maybe he's starting to rub off on me! >For this reason, it is probably unnecessary to represent the state >`present' at all, provided the semantics could reasonably require that >a feature which is absent is never described. Is that a reasonable >requirement? I think this is probably true. I can only come up with two potential problems: (1) In this case the state "present" is implied but this may not always be the case when states are not specified. For example: <character name="distribution"> <character name="country"> <state>Australia</state> </character> </character> A default state of "present" doesn't make much sense for the character "distribution". This may not be a serious problem but we should give it some thought. (2) I think the bigger problem is the flip-side: if a character isn't in the description then it must be "absent". We can't do it this way. >To me, the main thing that this kind of model implies is the need, in >some cases, to provide a thesaurus, e.g. to provide advice that if a >character (here `margin') has a subcharacter `teeth' then it may be >described as `serrate'. Is that bad? Or would a purely textual >description which just said "Leaf margins with forward-pointing teeth" >be deemed wrong absent the word "serrate" ? Adding a thesaurus would add way too much work and removing "serrate" would be too restrictive. >I think natural language parsing understanding is harder than natural >language production from structure. So I think there is less work to >go from data to description than the other way around. If true (and I think it is) then this would suggest a more DELTA-like structure and less of a text-markup structure for the representation. Steve

1 0

Re: It's How the Data will be Used that Counts
by Una Smith 04 Dec '01

04 Dec '01

Kevin Thiele wrote: >> I seem to be in a minority of one here again, but I'll continue to argue my >> case for a bit longer. Go for it. Defensiveness is a waste of everyone's time. >> 2. If we can effectively embed fully parsable data in a natural-language >> paragraph, why not? Jim Croft wrote: >becasue it creates mixed content (legal, but evil) where the data is >parsable, but not structured. [...] >In fact, DELTA already does this sort of thing by allowing liberal >appending and prepending <freeform comments> all over the place. While >this makes for quasireadable descriptions, authors often embed >interesting character data in the comments making it unusable by other >applications, even within the DELTA suite. [...] These <freeform comments> are natural-language paragraphs embedded in otherwise fully parsable data, which is the opposite of Kevin's case above. And much easier to handle, I think. I put interesting data in the comments for an extremely important reason: I know I don't have enough data to represent the probable diversity in the taxon (the item), and I do not want to understate the diversity. Better to leave the character coded as unknown, but record the data. At some point in the future, the data can be moved from the comments to where it belongs. -- Una Smith Los Alamos National Laboratory, Mailstop K-710, Los Alamos, NM 87545

1 0

It's How the Data will be Used that Counts
by unknown＠example.com 04 Dec '01

04 Dec '01

I've been giving Kevin's approach some thought and have the following comments: Kevin's original information flow model is too simplistic. A more realistic model would be something like: Text Descriptions | | Text Descriptions Phylogenetics |---> Structured Data --->| Phylogenetics Specimens | | Identification Tools (many) The sources are much more varied and are often group-specific. For example, invertebrates have very few good quality text descriptions (most are old, are in a range of languages (English, French, German, etc), vary greatly in style, quality, etc. etc) and the majority of invertebrates are currently undescribed (having 80% new taxa during a revision is common). Similarly, the outputs required vary greatly and in ways hard to predict. While text descriptions would seem to be a common requirement, they are in some ways "legacy" and may become less important in the future as applications (and users) become more sophisticated. We need to make sure we keep this range of uses in mind at all times. The Structured Data is the format we're talking about here. Don't know what that will look like yet (but see below). I think it's important to realise that applications will be used to move among these in almost all cases and only very rarely will people manipulate the data directly. It's also important to remember that XSLT is nothing more than an application. Some comments seem to imply that XSLT is part of XML, but it isn't. For example, from a BioLink perspective (and DELTA and probably LucID) the above model will need to be extended by adding: Text Descriptions | | Text Descriptions Phylogenetics |---> Structured Data --->| Phylogenetics Specimens | | | Identification Tools (many) | |<-- | BioLink |--> | DELTA | LucID Builder That is, applications will import Structured Data, manipulate it and spit it out again. Because of this I don't really think the details of the model matter too much, more that it is rich enough to represent all data of interest. ******** I've also been thinking about Kevin's latest example: "Leaf margins serrate with forward-pointing teeth" <feature name="leaf"> <feature name="margin"> <feature name = "teething shape"> <value>serrate</value> </feature> <feature name = "teeth orientation">with <value>forward-pointing</value>teeth </feature> </feature> </feature> First, it seems to me that "feature" is what taxonomists call "character" and "value" is "state". Being a traditionalist I'll switch back to this common terminology: "Leaf margins serrate with forward-pointing teeth" <character name="leaf"> <character name="margin"> <character name = "teething shape"> <state>serrate</state> </character> <character name = "teeth orientation"> <state>forward-pointing</state> </character> </character> </character> A couple of points: Kevin suggests a rule "<states>s cannot have <characters>s as siblings". But this is what DELTA calls Dependencies, it represents the state that controls a character. This would seem to be a good thing (and may be very important). Kevin's representation is too focused on text descriptions. A more complete representation might be: <character name="leaf"> <state>present</state> <character name="leaf margin"> <state>serrate</state> <character name = "tooth orientation"> <state>forward-pointing</state> </character> </character> </character> This allows us to directly extract: leaf = present leaf margin = serrate tooth orientation = forward-pointing This will be important for both identification tools and phylogenetics. Trying to recover this information from Kevin's representation should be possible but will require a number of assumptions be made about the data. This representation also captures dependencies (although this is an advanced feature we shouldn't be talking about yet). In my original, DELTA-centric model I used a <description> tag to try and capture the text description information separate from the <state> information. My thinking was that these two requirements/approaches/viewpoints are too distinct to cram together without falling into the same trap as the current DELTA Standard (which is a least-common denominator approach). The problem here is that the phrase "Leaf margins serrate with forward-pointing teeth" concerns 3 characters (leaf, margin and teeth) and 1 implied and 2 expressed states (present, serrate and forward pointing) with the characters being dependent (and therefore the context containing significant information - we know that 'teeth' have something to do with 'serrate' which has something to do with 'leaf margins' - the leaves being present because we're describing them). There's a lot of logic involved in parsing this. I can't think of a simple way of representing all this complex information without separating it at some level. Kevin's suggestion represents the text description and mine the underlying data, but neither works well for the other. Two steps forward, one step back. Sorry about that. Thanks, Steve

1 0

It's How the Data will be Used that Counts
by Robert A. (Bob) Morris 03 Dec '01

03 Dec '01

Wow, Steve, we are virtually in perfect agreement! A first! I have one small quibble---you actually raised it yourself though---and some suggestions Steve Shattuck writes: > ... > Kevin's representation is too focused on text descriptions. A more complete > representation might be: > > <character name="leaf"> > <state>present</state> > <character name="leaf margin"> > <state>serrate</state> > <character name = "tooth orientation"> > <state>forward-pointing</state> > </character> > </character> > </character> > > > This allows us to directly extract: > > leaf = present > leaf margin = serrate > tooth orientation = forward-pointing > > The problem here is that the phrase "Leaf margins serrate with > forward-pointing teeth" concerns 3 characters (leaf, margin and teeth) and 1 > implied and 2 expressed states (present, serrate and forward pointing) with > the characters being dependent (and therefore the context containing > significant information - we know that 'teeth' have something to do with > 'serrate' which has something to do with 'leaf margins' - the leaves being > present because we're describing them). For this reason, it is probably unnecessary to represent the state `present' at all, provided the semantics could reasonably require that a feature which is absent is never described. Is that a reasonable requirement? For example, in such a case to extract from the XML all taxa which have a leaf character in their description is not any harder for lacking <state>present</state>. In fact it is triffling bit easier. The desire for a `present' state perhaps comes from table-based character-by-state organization where it could be hard to distinguish whether the character is absent from the taxon or absent from the data. That distinction can be made moot here, perhaps. `presence' may be the only such state though. >There's a lot of logic involved in > parsing this. I can't think of a simple way of representing all this > complex information without separating it at some level. If we assume that there is a rigorous semantics to the effect that syntactically nested characters are always logically nested (and here we may need to return to "feature" if "character" is too dear to overload), is there a problem with this: <character name="leaf"> <character name="margin"> <character name="teeth"> <character name = "orientation"> <state>forward-pointing</state> </character> </character> </character> </character> To me, the main thing that this kind of model implies is the need, in some cases, to provide a thesaurus, e.g. to provide advice that if a character (here `margin') has a subcharacter `teeth' then it may be described as `serrate'. Is that bad? Or would a purely textual description which just said "Leaf margins with forward-pointing teeth" be deemed wrong absent the word "serrate" ? >Kevin's suggestion > represents the text description and mine the underlying data, agreed >but neither works well for the other. I think natural language parsing understanding is harder than natural language production from structure. So I think there is less work to go from data to description than the other way around. > > Two steps forward, one step back. Sorry about that. Nah, at most 1/2 step back. Is your middle name Zeno? > > Thanks, Steve > Bob Morris

1 0

Re: Thought and a Question
by Leigh Dodds 03 Dec '01

03 Dec '01

> | I'd like to suggest, again, that the solutions for the challenges > | should be presented as simple data models. Syntax can come later > | (and may fall naturally out of the model). > > Leigh - can you give us an example of your solution to challenge 1 then? Certainly. I'm trying to free up some time to devote a bit more effort to this, so I can back up my comments with some useful (hopefully!) contributions. Hope to have something later this week. Cheers, L.

1 0

Re: Challenge 1
by Cornelia Buchen-Osmond 03 Dec '01

03 Dec '01

I don't think that DELTA has given up too soon to "decide whether ease (as opposed to possibility) of producing documents is a more important design goal than ease of describing structures suitable for searching, integration with other structures such as those emerging from the Accessions groups efforts for specimen records, molecular data, and other data from sources other than those in a descriptive database". When starting with a database you often don't know, what is needed to suit best the demands. Text comments are very helpful to get to the first basis and the developer of the database has to integrate those comments later on into well structured language that is searchable. Dr Cornelia Büchen-Osmond Biosphere 2 Center 32540 S. Biosphere Road P.O. Box 689 Oracle, AZ 85623 U.S.A. Phone +1 (520) 896-5126 Fax +1 (520) 896-6429 email: buchen(a)bio2.columbia.edu ICTVdB - The Universal Virus Database http://www.ncbi.nlm.nih.gov/ICTVdb/ http://www.ictvdb.iacr.ac.uk/ http://www1.im.ac.cn/ictvdb/welcome.htm ----- Original Message ----- From: "Robert A. (Bob) Morris" <ram(a)CS.UMB.EDU> To: <TDWG-SDD(a)USOBI.ORG> Sent: Sunday, December 02, 2001 6:55 AM Subject: Re: Challenge 1 > I don't believe that minimizing markup is a desirable design goal. The > usual argument for it is that it makes the document easier for humans to read, > but the plain fact is that XML is rarely written or read by humans > except when they are either debugging or designing. If there really > are no natural tags to surround text in a particular case, it is > common to just surround it with <text> </text>. > > In this case, as is common, the tension arising is that between > articulating object structure and articulating document structure. As > I now understand DELTA, it devotes quite a bit of energy to ease of > producing documents, and maybe this group has to soon decide > whether ease (as opposed to possibility) of producing documents is a > more important design goal than ease of describing structures suitable > for searching, integration with other structures such as those > emerging from the Accessions groups efforts for specimen records, > molecular data, and other data from sources other than those in a > descriptive database. Resolving this tension with a compromise could > make /both/ enterprises hard instead of just one. My own bias, and I > think that of XML, is that it is easier to compute documents > given rigorous structure than the other way round. > > > > Kevin Thiele writes: > > Date: Sat, 1 Dec 2001 22:12:56 +1100 > > From: Kevin Thiele <kevin.thiele(a)bigpond.com> > > To: TDWG-SDD(a)usobi.org > > Subject: Re: Challenge 1 > > > > ----- Original Message ----- > > From: "Jim Croft" <jrc(a)ANBG.GOV.AU> > > To: <TDWG-SDD(a)USOBI.ORG> > > Sent: Saturday, December 01, 2001 8:30 AM > > Subject: Re: Challenge 1 > > > > > > | Kevin wrote: > > | >I've taken the route of marking up a textual description, using a minimum > > of > > | >tags. It seems to me that a description comprises a series of features > > with > > | >values. I've used mixed markup because I wanted to have the mimimum > > | >tagging and make maximum use of the text. > > | > > | Although XML allows mixed content such as Kevin's: > > | <Feature><Name>spines</Name>, not developing at each node, > > | <Feature Name="Length" MinValue="0">to c. > > | <MaxValue>1</MaxValue><Units>cm</Units> long</Feature> > > | </Feature> > > | and such a document can be validated against a schema, apart from being > > | untidy and inelegant, mixed content can pose certain problems when > > | attempted to be loaded into a relational database; this is probably not > > | going to impress people like Gregor. > > | > > | Can we agree that although mixed content XML is quite allowable, we are > > | going to try and avoid it in the SDD context? > > > > > > Hey, who said we should aim to make life easier for Gregor? I regard it as a > > life ambition to make things harder for software developers, especially > > those who insist on cramming mother nature kicking and screaming into a > > database! > > > > The mixed content of my suggestion is the thing I worry about most. But if > > there are advantages to mixed content (and it seems to me there are) then I > > think these should be weighed against the difficulty of dealing with it. > > Once we have more alternate solutions to challenge 1 we can deal with the > > advantages/disadvantages of particular solutions. I wouldn't like to > > preclude mixed content at this early stage - let's knock it on the head a > > bit further down the track. > > > > Cheers - k > > >

1 0

Re: Challenge 1
by Jim Croft 02 Dec '01

02 Dec '01

>The mixed content of my suggestion is the thing I worry about most. But if >there are advantages to mixed content (and it seems to me there are) then I >think these should be weighed against the difficulty of dealing with it. I am not yet convinced there are real advantages in mixed content, other than allowing the reproduction of a block of text closer to what was actually written in a prior publication. If the information can be represented structurally without resorting to mixed content then I think this is what we should do. However, it might be ok to defer banning mixed content outright until we are convinced we have a structure that caters for all our needs. My gut feeling is that the structure will not fully cater for our needs as long as there is mixed content. But we shall see... jim ~ Jim Croft ~ jrc(a)anbg.gov.au ~ 02-62465500 ~ www.anbg.gov.au/jrc/ ~

1 0

Re: Challenge 1
by Robert A. (Bob) Morris 02 Dec '01

02 Dec '01

I don't believe that minimizing markup is a desirable design goal. The usual argument for it is that it makes the document easier for humans to read, but the plain fact is that XML is rarely written or read by humans except when they are either debugging or designing. If there really are no natural tags to surround text in a particular case, it is common to just surround it with <text> </text>. In this case, as is common, the tension arising is that between articulating object structure and articulating document structure. As I now understand DELTA, it devotes quite a bit of energy to ease of producing documents, and maybe this group has to soon decide whether ease (as opposed to possibility) of producing documents is a more important design goal than ease of describing structures suitable for searching, integration with other structures such as those emerging from the Accessions groups efforts for specimen records, molecular data, and other data from sources other than those in a descriptive database. Resolving this tension with a compromise could make /both/ enterprises hard instead of just one. My own bias, and I think that of XML, is that it is easier to compute documents given rigorous structure than the other way round. Kevin Thiele writes: > Date: Sat, 1 Dec 2001 22:12:56 +1100 > From: Kevin Thiele <kevin.thiele(a)bigpond.com> > To: TDWG-SDD(a)usobi.org > Subject: Re: Challenge 1 > > ----- Original Message ----- > From: "Jim Croft" <jrc(a)ANBG.GOV.AU> > To: <TDWG-SDD(a)USOBI.ORG> > Sent: Saturday, December 01, 2001 8:30 AM > Subject: Re: Challenge 1 > > > | Kevin wrote: > | >I've taken the route of marking up a textual description, using a minimum > of > | >tags. It seems to me that a description comprises a series of features > with > | >values. I've used mixed markup because I wanted to have the mimimum > | >tagging and make maximum use of the text. > | > | Although XML allows mixed content such as Kevin's: > | <Feature><Name>spines</Name>, not developing at each node, > | <Feature Name="Length" MinValue="0">to c. > | <MaxValue>1</MaxValue><Units>cm</Units> long</Feature> > | </Feature> > | and such a document can be validated against a schema, apart from being > | untidy and inelegant, mixed content can pose certain problems when > | attempted to be loaded into a relational database; this is probably not > | going to impress people like Gregor. > | > | Can we agree that although mixed content XML is quite allowable, we are > | going to try and avoid it in the SDD context? > > > Hey, who said we should aim to make life easier for Gregor? I regard it as a > life ambition to make things harder for software developers, especially > those who insist on cramming mother nature kicking and screaming into a > database! > > The mixed content of my suggestion is the thing I worry about most. But if > there are advantages to mixed content (and it seems to me there are) then I > think these should be weighed against the difficulty of dealing with it. > Once we have more alternate solutions to challenge 1 we can deal with the > advantages/disadvantages of particular solutions. I wouldn't like to > preclude mixed content at this early stage - let's knock it on the head a > bit further down the track. > > Cheers - k >

1 0

Re: Challenge 1
by Kevin Thiele 01 Dec '01

01 Dec '01

----- Original Message ----- From: "Jim Croft" <jrc(a)ANBG.GOV.AU> To: <TDWG-SDD(a)USOBI.ORG> Sent: Saturday, December 01, 2001 8:30 AM Subject: Re: Challenge 1 | Kevin wrote: | >I've taken the route of marking up a textual description, using a minimum of | >tags. It seems to me that a description comprises a series of features with | >values. I've used mixed markup because I wanted to have the mimimum | >tagging and make maximum use of the text. | | Although XML allows mixed content such as Kevin's: | <Feature><Name>spines</Name>, not developing at each node, | <Feature Name="Length" MinValue="0">to c. | <MaxValue>1</MaxValue><Units>cm</Units> long</Feature> | </Feature> | and such a document can be validated against a schema, apart from being | untidy and inelegant, mixed content can pose certain problems when | attempted to be loaded into a relational database; this is probably not | going to impress people like Gregor. | | Can we agree that although mixed content XML is quite allowable, we are | going to try and avoid it in the SDD context? Hey, who said we should aim to make life easier for Gregor? I regard it as a life ambition to make things harder for software developers, especially those who insist on cramming mother nature kicking and screaming into a database! The mixed content of my suggestion is the thing I worry about most. But if there are advantages to mixed content (and it seems to me there are) then I think these should be weighed against the difficulty of dealing with it. Once we have more alternate solutions to challenge 1 we can deal with the advantages/disadvantages of particular solutions. I wouldn't like to preclude mixed content at this early stage - let's knock it on the head a bit further down the track. Cheers - k

1 0

Re: Thought and a Question
by Kevin Thiele 01 Dec '01

01 Dec '01

----- Original Message ----- From: "Leigh Dodds" <ldodds(a)INGENTA.COM> To: <TDWG-SDD(a)USOBI.ORG> Sent: Friday, November 30, 2001 8:27 PM Subject: Thought and a Question | I'd like to suggest, again, that the solutions for the challenges | should be presented as simple data models. Syntax can come later | (and may fall naturally out of the model). Leigh - can you give us an example of your solution to challenge 1 then?

1 0