November 1999 - tdwg-content

Markup of text descriptions Vs. structured data (GEN)
by Gregor Hagedorn 29 Nov '99

29 Nov '99

Jean-Marc Varel, Don Kirkup and others have repeatedly touched the problem of existing textual descriptions in their discussions contributed to this list. On the other hand, I myself belong to the group of DELTA/NEXUS minded people, who are concerned mostly with knowledge management of descriptive data that are created new. These issues should be seen quite separately, and that we need both 1. A markup language for text documents. This includes: - existing descriptions, captured by OCR or other means, where the markup would be manually (or automated/data mining?) - computer generated natural language descriptions that are published as electronic documents. A new CSIRO package could have a ToNat command that automatically adds the necessary, hidden markup code. 2. A data language for new observations, including repeated measurements or repeated observations of categorical data, e.g. shape of multiple leaves in a single specimen. Further, many structures for knowledge managements (data revision, annotation, quality control and assessment) need to be implemented here. The issues overlap, and it would be beneficial to use as much common syntax as possible, but fundamentally I believe them to be quite different. See also the separate post "Item description data in XML (XML)" for discussion of how to achieve this in XML and with XML examples. Gregor ---------------------------------------------------------- Inst. for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Gregor Hagedorn Net: G.Hagedorn(a)bba.de Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

(GEN) Schema independence
by Gregor Hagedorn 29 Nov '99

29 Nov '99

> > * I propose to have 3 XML Namespaces for our different XML > > vocabularies: > > o biological descriptions (generalities) > > o botany > > o zoology > > I'm sure you don't mean to leave out Mycology, Bryology, etc. Exactly (says a mycologist). Also, what about at least 20 different name spaces for protists, 50 or so for bacteria. Viruses, mycoplasms, viroids, ... Biological diversity is well beyond our current grasp. We should admit that we do NOT understand it in any way sufficiently. Even what we understand is obscured by terminoly inadequacies or concurrent, but not entirely synonymous terminology. From my own experience, I think that unifying character definition even among fairly closely related groups of a small group of the fungi can be a challenging task. Richard Pankhurst could probably step in here and explain why it is so difficult to even draw up a core consensus character set for all higher plants (an effort he is concerned with for several years now). The tendency in this discussion to assume that things like leaf, petiole, veins etc. should be directly coded into the standard worries me. The examples are fine for the purpose of discussion, but I hope that most people agree that the schema independence of DELTA (and principally also NEXUS, although NEXUS leaves most of the definition to free text in a published article) is a big achievement, and I would not like to sacrifice it. This does not mean that we should not try to unify character schemas, but it definitely should be a separate task. We need a general standard, and an applied standard that acutally uses a defined namespace. So, I believe we need: o Character schema definition language, expressed in XML-DTD or better XML-Schema - Researchers working on weird stuff like Tardigrada should not be required to wait until somebody creates a schema for them, but must go ahead and develop their own schema. - DELTA is, in part, such a language. o Character schema (an applied character definition, that acutally defines names, type, constraints, etc. for a set of characters), expressed in the new Character schema definition language. - The character definition of a DELTA data set is such a schema. - The thing to achieve would be, to express this in such a way that it becomes a usable XML Namespace definition, while still linking into the Character schema definition language! I could then be directly used by XML tools, but it would be developed with added value by the more relevant additional definition in the Character schema definition language! o An application to express item descriptions using these tools. - Example: DELTA ITEM DESCRIPTION. Thus I envision something like: + Character definition language, adding special metainformation to: + Standard XML Namespace definition, which, however, can be used independently Both together form a character schema of a new standard, which would be used in item descriptions. The item descriptions themselves have two components: + A markup language to mark existing (or computer generated) free textual descriptions + A data language, to exactly define the data used, including knowledge management support See also the separate post "Markup of text descriptions Vs. structured data (GEN)" Gregor ---------------------------------------------------------- Inst. for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Gregor Hagedorn Net: G.Hagedorn(a)bba.de Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Item description data in XML (XML)
by Gregor Hagedorn 29 Nov '99

29 Nov '99

In the separate post "Markup of text descriptions Vs. structured data (GEN)" I suggested that we need to distinguish between markup and data. Markup would be: <leaf>leaves usually obovate, 3 cm long</leaf> or <leaf>leaves usually <shape>obovate</shape>, <length>3 cm<length> long</leaf> Don suggested to encode the data element like: <leaf shape="obovate" length_mm="31"/> (NB: Please read the hardcoded namespace as an example only, see my separate "(GEN) Schema independence" post) As far as I understand XML, these attributes would be automatically hidden from view, which is fine for the markup purpose. However, I would prefer a coding method, which would allow comprehensive use of XML for the data definition part as well. Probably a stylesheet could determine whether only the free text with hidden markup is visible, only the data, or both? Any ideas, perhaps any technical suggestions, how to achieve that? Gregor ---------------------------------------------------------- Inst. for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Gregor Hagedorn Net: G.Hagedorn(a)bba.de Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Character types (RQT)
by Gregor Hagedorn 29 Nov '99

29 Nov '99

Stuart Poss wrote: > Are existing methods and this discussion > restricted to "morphological characters"? Personally I think that morpho-anatomical characters are not sufficiently different from other character like those mentioned in Stuarts post. I would want to discuss them here as well. Interesting things may, however, happen regarding character types. Besides an expanded list of character types (including date/time, ordered categorical: linear, cyclic, tree, or complex/reticulate) new types may appear. A new type that I am specially interested in, is the "alignment" type. A while ago on DELTA-L somebody commented that alignments can principally be treated as a group of categorical characters, just as they are in NEXUS. However, I believe this treatment is not adequate for the following reasons: o the homology of these characters is dependend on the alignment, especially: o adding new items (sequences) may change the homology assumptions within the remaining sequences o the naming/identification of characters would be extremely difficult to handle, because a base position depends on the alignment I believe that these special considerations require a special character type for aligned sequence data --- What other character types do you think are necessary? --- Whether images/audio/video material etc. should be considered character, I am doubtful. I currently prefer to summarize them together with formatted text documents and database access (information systems/Identification or query interfaces) or any other URLs as "external resources". I am not as clear about this as I would wish. One criterium could be that analytically accessible, well structured information should be considered a character (thus including restricted vocabulary text information, e.g. "lists") whereas only human usable resources should be treated different. Vector graphics lie somewhere in between... I appreciated the recent contributions about the parameterizations of shapes, I think this will be helpful! Gregor ---------------------------------------------------------- Inst. for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Gregor Hagedorn Net: G.Hagedorn(a)bba.de Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: Item description data in XML (XML)
by Leigh Dodds 29 Nov '99

29 Nov '99

> Don suggested to encode the data element like: > > <leaf shape="obovate" length_mm="31"/> > > (NB: Please read the hardcoded namespace as an example only, see > my separate "(GEN) Schema independence" post) > > As far as I understand XML, these attributes would be automatically > hidden from view, which is fine for the markup purpose. However, I > would prefer a coding method, which would allow comprehensive use of > XML for the data definition part as well. I'm not sure what you mean by 'hidden from view' - do you mean when rendered into a web browser? If so then I'd be wary about making assumptions as to what type of applications will be processing the markup. Using the DOM and/or SAX programming APIs I can get direct access to the attributes of an element and then process them just as I could any other markup. My XML based application may not render the data at all - it might just use the data to perform some other functionality - lets not get too fixed up on the visual representation. > Probably a stylesheet could determine whether only the free text with > hidden markup is visible, only the data, or both? > Any ideas, perhaps any technical suggestions, how to achieve that? Again, apologies if I mis-read your intent... An XSL stylesheet defines how to render an XML document into 'Formatting Objects' which are then handled by the renderer as it requires. One such renderer may be a web browser. Another might be a PDF document generator. XSL is based on XSLT which is a means of taking an XML document and producing another document from it (not necessarily XML). XSLT currently supports three output methods - XML, plain-text and HTML. XSLT is commonly used to take XML and render it into HTML for the web - useful because there is limited support for XML in browsers. IE5.0 for example has an XSL implementation which is out of date, and Netscape lacks one at all. Now to go back to your question - how the XML is handled depends on the application. One application might be generating plain-text descriptions of the data, or indeed backwards compatible files (I give an XSLT stylesheet for generating DELTA character files from XDELTA files at : http://www.bath.ac.uk/~ccslrd/delta/) Another application might render the XML directly into a browser, and might use another suitable stylesheet. So the first thing to note is that your stylesheet is going to be determined by what kind of output you're expecting - I wouldn't expect a 'default' stylesheet at all. Instead there will be different stylesheets for different contexts (web browser, PDF, plain-text, etc, etc). Secondly I might not be generating anything visible at all. I can well imagine an application that will take an XML document and from the data within produce (say) a taxonomic tree or tress of that data. Here I wouldn't use a stylesheet, I'd simply process the data directly. Let me know if I can provide any more info on the above. Cheers, L.

1 0

Re: Character types (RQT)
by Leigh Dodds 29 Nov '99

29 Nov '99

> Whether images/audio/video material etc. should be considered > character, I am doubtful. I currently prefer to summarize them > together with formatted text documents and database access > (information systems/Identification or query interfaces) or any other > URLs as "external resources". I'd agree here. From the examples I've seen of DELTA files, using characters (or at least their syntax) to describe images is just a convenience - they're not naturally part of the model. images/audio/video are supporting resources and certainly wouldn't be embedded within the format (although this is possible). I could envisage an audio clip being useful in identifying an item, but a textual description to accompany the resource ought to be a requirement. This means that anyone using the format 'offline' doesn't need to access some resource storage (i.e. the net) to make a successful identification. > Vector graphics lie somewhere in between... I appreciated the recent > contributions about the parameterizations of shapes, I think this > will be helpful! Yes, rendering a graphical representation of (say) a leaf onto the screen would be a good application for vector graphics. However again some additional text describing the image/shape would be useful - for those applications that don't support vector graphics, or indeed visually impaired users. L.

1 0

Re: (XML)Pedagogy
by Gregor Hagedorn 29 Nov '99

29 Nov '99

Thanks for your talk about XML. I have several questions: o Are Vocabulary and Namespace synonymous terms? o What is XML-DATA? o Reading about RDF and Dublin Core as RDF application I was left confused as to what acutally is a resource. It usually seems to be assumed that it is a document, but this impression may only be due to the examples. Thus, in a file with several item descriptions could: - Each item be a resource, for which a creator/editor etc. is described using DublinCore? - Could even the characters information within an item have different authors? (And if yes: Is it wise to do this?) - Is there a limit to a hierarchy of RDF objects in an XML file? > You can copy-and-paste this into an example.html file, and see it in > Internet Explorer 5, or Mozilla, or any XHTML+CSS1 compliant browser. Note: I assume that Mozilla does not apply to Netscape 4.7, which did not seem to be able to display the code you appended > * how and why mix several vocabularies, I would be very interested in this one. thanks Gregor ---------------------------------------------------------- Inst. for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Gregor Hagedorn Net: G.Hagedorn(a)bba.de Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: (XML)Pedagogy
by Leigh Dodds 29 Nov '99

29 Nov '99

> o Are Vocabulary and Namespace synonymous terms? Not quite, and I'd actually say that the term 'vocabulary' might be a little off base - I'd prefer 'schema' based on the previous example, and common usage: A vocabulary (or schema) is the syntax for your XML file. It defines what are the legal elements and attributes, and how they can be combined. A namespace (declaration) indicates that a particular XML element or attribute belongs to a particular schema (vocabulary). This means that you can build an XML document based on more than one schema (vocabulary) and qualify the elements/attributes to denote which schema they derive from. This is obviously useful if two schema have elements/attributes of the same name - you can explicitly state which version you are using. The Namespace Recommendation can be found at : http://www.w3.org/TR/REC-xml-names/ > o What is XML-DATA? See http://www.w3.org/TR/1998/NOTE-XML-data-0105/ XML-DATA is one of many proposed schema languages which includes XML Schemas, SOX and DSD. I'd suggest that XML Schemas is likely to become the standard simply because its further along the review process. > > * how and why mix several vocabularies, > > I would be very interested in this one. As are a lot of XML developers! Its hoped that the next version of the Schemas spec will address these issues. http://www.lists.ic.ac.uk/hypermail/xml-dev/xml-dev-Nov-1999/0916.html Cheers, L.

1 0

Item description data in XML (XML)
by Robert A. (Bob) Morris 29 Nov '99

29 Nov '99

Gregor Hagedorn writes: >[...] > Probably a stylesheet could determine whether only the free text with > hidden markup is visible, only the data, or both? > > Any ideas, perhaps any technical suggestions, how to achieve that? > XSLT, now an official W3 Recommendation, is the language for transforming XML documents into other XML documents. From the point of view of XSLT, document nodes include text, XML elements, element attributes, comments, and a few other things. This makes it particularly easy to hide or reveal either the text or the markup, or the attributes in various combinations. Bob Morris

1 0

(RQT) 250 000 species in my hand
by Jean-Marc Vanel 28 Nov '99

28 Nov '99

In a CD of course. And not in 5 years... There is a strong necessity to have an exhaustive coverage of species. A telephone directory with only 1/100 of the persons is useless. Science is not only a matter of doing publications to advance careers. It has social implications. If an information of high quality is available on the Net, it will greatly advance the cause of nature conservation. It will also advance the cause of descriptive biology and field biology, and help to raise funds (which go to molecular biology mainly). The paradox is that there is a growing sympathy in the general public for ecology, nature conservation, biodiversity, etc. But at the same time there is growing ignorance in Natural History. However there is a hunger to learn and understand that isn't satisfied by books. But the answers aren't at hand. With current floras and descriptions properly computerized, with the web, and some logic programming and image pattern recognition, we can remedy that. It seems that descriptive biology needs help from computer scientists, not only database and knowledge management, but also image processing (see other posts about "Types of data"). More on those matters at http://jmvanel.free.fr/ . Cheers Jean-Marc Vanel

1 0