- tdwg-content - lists.tdwg.org

Re: is there an "xml-include"
by Douglas Trainor 21 Nov '01

21 Nov '01

Amen. I'm not a taxonomist, but I suggest that several folks on this list take some of their current character data and make an example out of it in XML and comment on the example. It need not be perfect. This could quickly educate others about the strengths and potential weaknesses of an approach with a specific examplar. XML will make it very nice to move data between different applications, or between component parts of a large application. I just want to write nifty plant/insect applications against the XML-wrapped character data. Going right off for a perfect Holy Grail has sabotaged other group projects in the past. One of the best examples is how Algol language folks got into bitter debates about when an object is a chair and when it's a table. douglas Steve Shattuck wrote: > There seems to be a number of problems being addressed in the current > discussion: > > 1) Gregor's original post asked for a solution ("how do I do an include in > XML") without posing a problem ("how do I maintain a separate global > character list and link a number of independent taxon descriptions to it"). > I would suggest that using an "include" is only one of a range of solutions > to this problem. And if this isn't the problem then we should revisit it > before spending too much time coming up with a solution. > > 2) Whether XML is hierarchical or relational is of little importance in > practice. All major relational databases can suck in and spit out XML - > it's a non-issue from an implementation standpoint (which is what we are > ultimately concerned with here). IDs and RefIDs are primary and foreign > keys - the mapping is one-to-one if you want to structure the XML that way. > > 3) Leigh is spot-on when he says "define a model for the data" as this is > "separate to the details of its [the model's] syntax". The syntax (= XML > model) will be useless if it doesn't manage the information that's important > to us no matter how "rigorous" or "technically accurate" it is. > > 4) At the Sydney meeting I thought we agreed to restart this discussion with > specific examples based on real-life characters (in the biological sense). > This current thread would seem to be very much a continuation of the one > started 2 years ago: 80% focus on solutions (largely round XML syntax) and > 20% on specific problems (addressing business needs). I fear we'll end up > the same place we were before. > > Thanks, Steve > > Steve Shattuck > CSIRO Entomology > biolink(a)ento.csiro.au > > P.S. - A solution to Gregor's problem would be to use unique identifiers > (GUIDs) in his global character list. Any one using this list would then > include these identifiers as part of taxon descriptions along with metadata > (a citation) to find where the global list is housed. If you then want to > combine separate description datasets you can compare the character > identifiers: if they are the same then it's the same character, if they are > different then it's not the same character. It's a simple, foolproof > solution that's easy to implement technically (whether people will actually > do this is a completely different matter ;-) ). > > P.P.S. - I'm well aware that I have the power to direct this discussion by > actually doing what I suggest in No. 4 above. And I really, really intend > to do it. I just have one more meeting to get through, and then another 10 > emails, and then ....

1 0

Re: is there an "xml-include"
by Leigh Dodds 21 Nov '01

21 Nov '01

> I think this comes closest. Can you point me to information on > XInclude? As you say, it seems unclear whether a validating parser, > say msxml 4.0 would be able to check such id/idrefs. http://www.w3.org/TR/xinclude XInclude in Java: http://www.ibiblio.org/xml/XInclude/ Cheers, L.

1 0

Modelling and XML
by Leigh Dodds 21 Nov '01

21 Nov '01

As an outside, but interested, observer I'm keen to contribute to this discussion in any way that seems useful. E.g. technical help with XML. Judging by the recent discussion there is progress being made on this front. Is there any publically available documentation? I'll also interject 2p/euro/cents at this point: I believe I'd tackle this problem firstly from a modelling perspective: can we agree upon, and define a model for the data that we're capturing. It seems like there is a conceptual model inherent in the data currently captured by DELTA and other formats, that is separate to the details of its syntax. Having agreed a particular model, a suitable syntax for expressing that model would be the logical next step. XML seems like a good fit for this, and, as I've noted elsewhere, can serialise graphs of information. Cheers, L. -- Leigh Dodds, Research Group, Ingenta | "Pluralitas non est ponenda http://weblogs.userland.com/eclectic | sine necessitate" http://www.xml.com/pub/xmldeviant | -- William of Ockham

1 0

Re: is there an "xml-include"
by Leigh Dodds 21 Nov '01

21 Nov '01

At risk of distracting from the discussion at hand: > If you feel that descriptive data can only be modeled with > relations, then you can never get XML to work for you. It is not relational, it is strictly > hierarcical. It does not and, by design, cannot support the relational model. Do not > confuse XML with a database. It is -not- designed to be a database. The few databases that > use XML are all hierarcical XML is designed to model the structure of > documents. If you want to create a database, use a database > language. XML is what you use in the report. Well strictly speaking XML can be used to model/serialise graph structures, so while it is hierarchical, and the syntax does require a single root, there's no reason why this can't become a graph (or other) structure once parsed. There's been a lot of activity (unsurprisingly) in serialising XML into and out of databases. In general it's not an easy problem to solve, but it can be done. Also, I'd have to disagree with the statement that 'the few databases that use XML are all hierarchical'. Current XML-aware databases fall into two categories: XML-Enabled Databases (i.e. relational databases that have added XML features) and Native XML Databases (i.e. those specifically designed to store XML). Databases in the first category do tend to be primarily relational in nature. Those in the second are a mixture: they model the XML document internally, and this might be achieved using a relational model. Cheers, L. -- Leigh Dodds, Research Group, Ingenta | "Pluralitas non est ponenda http://weblogs.userland.com/eclectic | sine necessitate" http://www.xml.com/pub/xmldeviant | -- William of Ockham

1 0

Re: is there an "xml-include"
by Robert A. (Bob) Morris 20 Nov '01

20 Nov '01

Leigh Dodds writes: > Date: Wed, 21 Nov 2001 09:15:02 -0000 > From: Leigh Dodds <ldodds(a)INGENTA.COM> > To: TDWG-SDD(a)usobi.org > Subject: Re: is there an "xml-include" > > At risk of distracting from the discussion at hand: > >[...] > Well strictly speaking XML can be used to model/serialise graph structures, > so while it is hierarchical, and the syntax does require a single root, > there's > no reason why this can't become a graph (or other) structure once parsed. > > There's been a lot of activity (unsurprisingly) in serialising XML into and > out of databases. In general it's not an easy problem to solve, but it > can be done. People interested in how easily it can always be done, but why it is hard to do it well in general, might consult S. Abiteboul, P. Buneman, D. Suciu, "Data on the Web. From Relations to Semistructured Data and XML", Morgan Kaufmann Publishers, 2000, especially pp. 172 ff.

1 0

Re: XML: is there an "xml-include"
by Guillaume Rousse 20 Nov '01

20 Nov '01

Ainsi parlait Gregor Hagedorn : > If character are defined with an id (numeric or character) in one > file, and the item descriptions use these ids through idref or > through xml-schema keyref means: > > How can a validating parser validate the schema, including keyrefs, > without having to include the entire character definition in each of > 1000s of taxon description xml files? Can the ids for idref/keyref be > declared to be in a separate file? > > I could not find a standard xml-include command. I know there is one > defined for dtds, but that is all I could find. > > Can the xml experts help? I'll try to summarize the discussion we had during the field trip, maybe it will be clearer for everyone. You want to have a descriptors file (the model), validated against a descriptors schema. And a taxons file, validated againt a taxons schema. But you also want to ensure coherency between the model and the taxons, such as referencing the correct descriptors, or the correct descriptors states. One way would be to use XSLT to performs this additional coherency enforcement. You can include the descriptors file in the taxons file before validating it, you can search for aberrant xpath statement result, etc... The other way i would suggest would be to drop the static taxons schema, and dynamically generate it from the descriptors file using XSLT. Then you'll have one-step validation. But no more generic descriptive model. BTW, the second approch is the one used by IKBS project (http://www.univ-reunion.fr/ikbs), even if they don't use XML. -- Guillaume Rousse <rousse(a)ccr.jussieu.fr> GPG key http://lis.snv.jussieu.fr/~rousse/gpgkey.html

1 0

Re: is there an "xml-include"
by Gregor Hagedorn 20 Nov '01

20 Nov '01

> This is a means to separate out a schema into multiple files, > if I understand Gregor correctly he's asking for a way to test for > id/keyref uniqueness across multiple files? Yes, and data files, not schema/dtd. One xml data file defines ids, the other points to these ids. In the case of descriptive data, 1000s of item description files should all point to the same, centrally defined character definitions. > At a higher level there is XInclude which allows for including > content from one document into another. Not sure on > current support for this however. I think this comes closest. Can you point me to information on XInclude? As you say, it seems unclear whether a validating parser, say msxml 4.0 would be able to check such id/idrefs. Gregor ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: XML: is there an "xml-include"
by Gregor Hagedorn 20 Nov '01

20 Nov '01

> Yes. See > http://www.w3.org/TR/xmlschema-0/#SchemaInMultDocs I think this is not possible. If I understand correctly, you are referring to schema, not idref in one xml document referring to id defined in another document. The schema would define the fact that two attribute (say cid and cidref) have to be interpreted as key and keyref, but not the actual values. ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: is there an "xml-include"
by Barringer, Kerry 20 Nov '01

20 Nov '01

Gregor, We disagree and I'd like to get to the reason why because I am sure that both of us are not fully understanding what the other is saying. I agree that a character name is not necessarily unique across different taxa. A character is what a person defines it to be. But I don't understand what you mean when you say that the name cannot stand for the character, that only a code can. This seems absurd. I don't see the difference between my 'name' and your 'code', especially when your code is just a compact coding that refers to a name. In your haircolor example, I think you are misunderstanding the hierarcical nature of XML coding. Each character is part of another, which all eventually are part of a 'root,' in our case probably a plant. So the character 'Plant' has the characters 'stems.' 'leaves,' etc. 'Leaves" may contain the character 'trichomes' which may contain the character 'color.' If I understand what you wrote, your criticism applies to a relational model where the character of 'aircolor' might be used for leaf trichomes, fine enations inside a fruit, or the body covering of mammals. It is impossible to define a character this way in an XML file because it is strictly hierarcical, not relational. If you feel that descriptive data can only be modeled with relations, then you can never get XML to work for you. It is not relational, it is strictly hierarcical. It does not and, by design, cannot support the relational model. Do not confuse XML with a database. It is -not- designed to be a database. The few databases that use XML are all hierarcical XML is designed to model the structure of documents. If you want to create a database, use a database language. XML is what you use in the report. My feeling is that the hierarcical nature of XML might be suitable for plant descriptions. I know it does a good job coding taxomomic papers and taxonomic nomenclature. It is less suitable, though still workable, for specimen data, which is easier to model with relational tables. To use XML though, I think we must look outside the DELTA model of character coding. XML won't improve the DELTA model or fix its faults. We must also look beyond the database model. Plant descriptions are highly structured documents. XML can model that structure. Finally, we have to get beyond the wordiness of XML files. They are designed to be that way. we can only live with it. Let me know if this makes sense and be sure to correct me if you feel I am being too stupid. all the best, Kerry -----Original Message----- From: Gregor Hagedorn [mailto:G.Hagedorn@BBA.DE] Sent: Tuesday, November 20, 2001 10:31 AM To: TDWG-SDD(a)USOBI.ORG Subject: Re: is there an "xml-include" Dear Kerry, I can not see how what you propose could work. The name of a character is not necessarily unique across different taxa, and often not even within a taxon. Further the character concept and its names (technical, laymen, English, German, etc.) are a 1:n relation. The name can not stand for the character, only a code can. I do not care whether this code is characters or numbers, but I believe it is a mistake to think if something is <haircolor></haircolor> to take the code at face value and assume you know what are hairs and what is color. Hairs are quite different things in plants, animals, or fungi, and color needs information whether it is a code from a color comparison chart, or an undefined term like "red". I think I am sticking less with a DELTA storage optimization model, but with a relational information model, which is what I have been using for all my projects. The relational model allows language independence. How can you preserve that, without having unique codes that lead to the definition of a character? That does not mean, that a free text description in some language, say Chinese, may be present, in addition to the data. That is why I am thinking of attributes, not element data. Gregor ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: is there an "xml-include"
by Gregor Hagedorn 20 Nov '01

20 Nov '01

Dear Kerry, I can not see how what you propose could work. The name of a character is not necessarily unique across different taxa, and often not even within a taxon. Further the character concept and its names (technical, laymen, English, German, etc.) are a 1:n relation. The name can not stand for the character, only a code can. I do not care whether this code is characters or numbers, but I believe it is a mistake to think if something is <haircolor></haircolor> to take the code at face value and assume you know what are hairs and what is color. Hairs are quite different things in plants, animals, or fungi, and color needs information whether it is a code from a color comparison chart, or an undefined term like "red". I think I am sticking less with a DELTA storage optimization model, but with a relational information model, which is what I have been using for all my projects. The relational model allows language independence. How can you preserve that, without having unique codes that lead to the definition of a character? That does not mean, that a free text description in some language, say Chinese, may be present, in addition to the data. That is why I am thinking of attributes, not element data. Gregor ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0