- tdwg-content - lists.tdwg.org

Re: is there an "xml-include"
by unknown＠example.com 22 Nov '01

22 Nov '01

There seems to be a number of problems being addressed in the current discussion: 1) Gregor's original post asked for a solution ("how do I do an include in XML") without posing a problem ("how do I maintain a separate global character list and link a number of independent taxon descriptions to it"). I would suggest that using an "include" is only one of a range of solutions to this problem. And if this isn't the problem then we should revisit it before spending too much time coming up with a solution. 2) Whether XML is hierarchical or relational is of little importance in practice. All major relational databases can suck in and spit out XML - it's a non-issue from an implementation standpoint (which is what we are ultimately concerned with here). IDs and RefIDs are primary and foreign keys - the mapping is one-to-one if you want to structure the XML that way. 3) Leigh is spot-on when he says "define a model for the data" as this is "separate to the details of its [the model's] syntax". The syntax (= XML model) will be useless if it doesn't manage the information that's important to us no matter how "rigorous" or "technically accurate" it is. 4) At the Sydney meeting I thought we agreed to restart this discussion with specific examples based on real-life characters (in the biological sense). This current thread would seem to be very much a continuation of the one started 2 years ago: 80% focus on solutions (largely round XML syntax) and 20% on specific problems (addressing business needs). I fear we'll end up the same place we were before. Thanks, Steve Steve Shattuck CSIRO Entomology biolink(a)ento.csiro.au P.S. - A solution to Gregor's problem would be to use unique identifiers (GUIDs) in his global character list. Any one using this list would then include these identifiers as part of taxon descriptions along with metadata (a citation) to find where the global list is housed. If you then want to combine separate description datasets you can compare the character identifiers: if they are the same then it's the same character, if they are different then it's not the same character. It's a simple, foolproof solution that's easy to implement technically (whether people will actually do this is a completely different matter ;-) ). P.P.S. - I'm well aware that I have the power to direct this discussion by actually doing what I suggest in No. 4 above. And I really, really intend to do it. I just have one more meeting to get through, and then another 10 emails, and then ....

1 0

Re: is there an "xml-include"
by Robert A. (Bob) Morris 21 Nov '01

21 Nov '01

Steve Shattuck writes: > Date: Thu, 22 Nov 2001 10:41:55 +1100 >[...] > 4) At the Sydney meeting I thought we agreed to restart this discussion with > specific examples based on real-life characters (in the biological sense). > This current thread would seem to be very much a continuation of the one > started 2 years ago: 80% focus on solutions (largely round XML syntax) and > 20% on specific problems (addressing business needs). I fear we'll end up > the same place we were before. Amen. More explicitly, what was posed was to provide "data challenges", i.e. (as I understood) problems about characters that the poser feels important to model. Implicit (as I understood) in that discussion---and in almost all of this recent thread---is that a particular representation (e.g. XML) is NOT what is initially desired. If, indeed, my understandings were correct, and if a sufficiently inclusive set of challenges were identified, what would then be accomplished sets the stage exactly for the modeling Leigh urges. Further, this complies with one of the admonitions in Rob Atkinson's talk to first find out what you are trying to make a standard about, i.e.: <not> <adage>XML is the answer. What is the question?</adage> </not> Finally, I would paraphrase what I understood the subgroup meeting arrived at: The first job is to identify with these data challenges the strengths and the limitations of what DELTA-motivated applications can accomplish, rather than throw out the experience of DELTA and start over. Probably Gregor's meeting minutes will reveal whether that understanding is correct and perhaps how it came to be. --Bob Morris

1 0

Re: is there an "xml-include"
by Douglas Trainor 21 Nov '01

21 Nov '01

Amen. I'm not a taxonomist, but I suggest that several folks on this list take some of their current character data and make an example out of it in XML and comment on the example. It need not be perfect. This could quickly educate others about the strengths and potential weaknesses of an approach with a specific examplar. XML will make it very nice to move data between different applications, or between component parts of a large application. I just want to write nifty plant/insect applications against the XML-wrapped character data. Going right off for a perfect Holy Grail has sabotaged other group projects in the past. One of the best examples is how Algol language folks got into bitter debates about when an object is a chair and when it's a table. douglas Steve Shattuck wrote: > There seems to be a number of problems being addressed in the current > discussion: > > 1) Gregor's original post asked for a solution ("how do I do an include in > XML") without posing a problem ("how do I maintain a separate global > character list and link a number of independent taxon descriptions to it"). > I would suggest that using an "include" is only one of a range of solutions > to this problem. And if this isn't the problem then we should revisit it > before spending too much time coming up with a solution. > > 2) Whether XML is hierarchical or relational is of little importance in > practice. All major relational databases can suck in and spit out XML - > it's a non-issue from an implementation standpoint (which is what we are > ultimately concerned with here). IDs and RefIDs are primary and foreign > keys - the mapping is one-to-one if you want to structure the XML that way. > > 3) Leigh is spot-on when he says "define a model for the data" as this is > "separate to the details of its [the model's] syntax". The syntax (= XML > model) will be useless if it doesn't manage the information that's important > to us no matter how "rigorous" or "technically accurate" it is. > > 4) At the Sydney meeting I thought we agreed to restart this discussion with > specific examples based on real-life characters (in the biological sense). > This current thread would seem to be very much a continuation of the one > started 2 years ago: 80% focus on solutions (largely round XML syntax) and > 20% on specific problems (addressing business needs). I fear we'll end up > the same place we were before. > > Thanks, Steve > > Steve Shattuck > CSIRO Entomology > biolink(a)ento.csiro.au > > P.S. - A solution to Gregor's problem would be to use unique identifiers > (GUIDs) in his global character list. Any one using this list would then > include these identifiers as part of taxon descriptions along with metadata > (a citation) to find where the global list is housed. If you then want to > combine separate description datasets you can compare the character > identifiers: if they are the same then it's the same character, if they are > different then it's not the same character. It's a simple, foolproof > solution that's easy to implement technically (whether people will actually > do this is a completely different matter ;-) ). > > P.P.S. - I'm well aware that I have the power to direct this discussion by > actually doing what I suggest in No. 4 above. And I really, really intend > to do it. I just have one more meeting to get through, and then another 10 > emails, and then ....

1 0

Re: is there an "xml-include"
by Leigh Dodds 21 Nov '01

21 Nov '01

> I think this comes closest. Can you point me to information on > XInclude? As you say, it seems unclear whether a validating parser, > say msxml 4.0 would be able to check such id/idrefs. http://www.w3.org/TR/xinclude XInclude in Java: http://www.ibiblio.org/xml/XInclude/ Cheers, L.

1 0

Modelling and XML
by Leigh Dodds 21 Nov '01

21 Nov '01

As an outside, but interested, observer I'm keen to contribute to this discussion in any way that seems useful. E.g. technical help with XML. Judging by the recent discussion there is progress being made on this front. Is there any publically available documentation? I'll also interject 2p/euro/cents at this point: I believe I'd tackle this problem firstly from a modelling perspective: can we agree upon, and define a model for the data that we're capturing. It seems like there is a conceptual model inherent in the data currently captured by DELTA and other formats, that is separate to the details of its syntax. Having agreed a particular model, a suitable syntax for expressing that model would be the logical next step. XML seems like a good fit for this, and, as I've noted elsewhere, can serialise graphs of information. Cheers, L. -- Leigh Dodds, Research Group, Ingenta | "Pluralitas non est ponenda http://weblogs.userland.com/eclectic | sine necessitate" http://www.xml.com/pub/xmldeviant | -- William of Ockham

1 0

Re: is there an "xml-include"
by Leigh Dodds 21 Nov '01

21 Nov '01

At risk of distracting from the discussion at hand: > If you feel that descriptive data can only be modeled with > relations, then you can never get XML to work for you. It is not relational, it is strictly > hierarcical. It does not and, by design, cannot support the relational model. Do not > confuse XML with a database. It is -not- designed to be a database. The few databases that > use XML are all hierarcical XML is designed to model the structure of > documents. If you want to create a database, use a database > language. XML is what you use in the report. Well strictly speaking XML can be used to model/serialise graph structures, so while it is hierarchical, and the syntax does require a single root, there's no reason why this can't become a graph (or other) structure once parsed. There's been a lot of activity (unsurprisingly) in serialising XML into and out of databases. In general it's not an easy problem to solve, but it can be done. Also, I'd have to disagree with the statement that 'the few databases that use XML are all hierarchical'. Current XML-aware databases fall into two categories: XML-Enabled Databases (i.e. relational databases that have added XML features) and Native XML Databases (i.e. those specifically designed to store XML). Databases in the first category do tend to be primarily relational in nature. Those in the second are a mixture: they model the XML document internally, and this might be achieved using a relational model. Cheers, L. -- Leigh Dodds, Research Group, Ingenta | "Pluralitas non est ponenda http://weblogs.userland.com/eclectic | sine necessitate" http://www.xml.com/pub/xmldeviant | -- William of Ockham

1 0

Re: is there an "xml-include"
by Robert A. (Bob) Morris 21 Nov '01

21 Nov '01

Leigh Dodds writes: > Date: Wed, 21 Nov 2001 09:15:02 -0000 > From: Leigh Dodds <ldodds(a)INGENTA.COM> > To: TDWG-SDD(a)usobi.org > Subject: Re: is there an "xml-include" > > At risk of distracting from the discussion at hand: > >[...] > Well strictly speaking XML can be used to model/serialise graph structures, > so while it is hierarchical, and the syntax does require a single root, > there's > no reason why this can't become a graph (or other) structure once parsed. > > There's been a lot of activity (unsurprisingly) in serialising XML into and > out of databases. In general it's not an easy problem to solve, but it > can be done. People interested in how easily it can always be done, but why it is hard to do it well in general, might consult S. Abiteboul, P. Buneman, D. Suciu, "Data on the Web. From Relations to Semistructured Data and XML", Morgan Kaufmann Publishers, 2000, especially pp. 172 ff.

1 0

Re: XML: is there an "xml-include"
by Guillaume Rousse 20 Nov '01

20 Nov '01

Ainsi parlait Gregor Hagedorn : > If character are defined with an id (numeric or character) in one > file, and the item descriptions use these ids through idref or > through xml-schema keyref means: > > How can a validating parser validate the schema, including keyrefs, > without having to include the entire character definition in each of > 1000s of taxon description xml files? Can the ids for idref/keyref be > declared to be in a separate file? > > I could not find a standard xml-include command. I know there is one > defined for dtds, but that is all I could find. > > Can the xml experts help? I'll try to summarize the discussion we had during the field trip, maybe it will be clearer for everyone. You want to have a descriptors file (the model), validated against a descriptors schema. And a taxons file, validated againt a taxons schema. But you also want to ensure coherency between the model and the taxons, such as referencing the correct descriptors, or the correct descriptors states. One way would be to use XSLT to performs this additional coherency enforcement. You can include the descriptors file in the taxons file before validating it, you can search for aberrant xpath statement result, etc... The other way i would suggest would be to drop the static taxons schema, and dynamically generate it from the descriptors file using XSLT. Then you'll have one-step validation. But no more generic descriptive model. BTW, the second approch is the one used by IKBS project (http://www.univ-reunion.fr/ikbs) even if they don't use XML. -- Guillaume Rousse <rousse(a)ccr.jussieu.fr> GPG key http://lis.snv.jussieu.fr/~rousse/gpgkey.html

1 0

Re: is there an "xml-include"
by Gregor Hagedorn 20 Nov '01

20 Nov '01

> This is a means to separate out a schema into multiple files, > if I understand Gregor correctly he's asking for a way to test for > id/keyref uniqueness across multiple files? Yes, and data files, not schema/dtd. One xml data file defines ids, the other points to these ids. In the case of descriptive data, 1000s of item description files should all point to the same, centrally defined character definitions. > At a higher level there is XInclude which allows for including > content from one document into another. Not sure on > current support for this however. I think this comes closest. Can you point me to information on XInclude? As you say, it seems unclear whether a validating parser, say msxml 4.0 would be able to check such id/idrefs. Gregor ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: XML: is there an "xml-include"
by Gregor Hagedorn 20 Nov '01

20 Nov '01

> Yes. See > http://www.w3.org/TR/xmlschema-0/#SchemaInMultDocs I think this is not possible. If I understand correctly, you are referring to schema, not idref in one xml document referring to id defined in another document. The schema would define the fact that two attribute (say cid and cidref) have to be interpreted as key and keyref, but not the actual values. ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0