November 2001 - tdwg-content

Re: XML: is there an "xml-include"
by Guillaume Rousse 20 Nov '01

20 Nov '01

Ainsi parlait Gregor Hagedorn : > If character are defined with an id (numeric or character) in one > file, and the item descriptions use these ids through idref or > through xml-schema keyref means: > > How can a validating parser validate the schema, including keyrefs, > without having to include the entire character definition in each of > 1000s of taxon description xml files? Can the ids for idref/keyref be > declared to be in a separate file? > > I could not find a standard xml-include command. I know there is one > defined for dtds, but that is all I could find. > > Can the xml experts help? I'll try to summarize the discussion we had during the field trip, maybe it will be clearer for everyone. You want to have a descriptors file (the model), validated against a descriptors schema. And a taxons file, validated againt a taxons schema. But you also want to ensure coherency between the model and the taxons, such as referencing the correct descriptors, or the correct descriptors states. One way would be to use XSLT to performs this additional coherency enforcement. You can include the descriptors file in the taxons file before validating it, you can search for aberrant xpath statement result, etc... The other way i would suggest would be to drop the static taxons schema, and dynamically generate it from the descriptors file using XSLT. Then you'll have one-step validation. But no more generic descriptive model. BTW, the second approch is the one used by IKBS project (http://www.univ-reunion.fr/ikbs), even if they don't use XML. -- Guillaume Rousse <rousse(a)ccr.jussieu.fr> GPG key http://lis.snv.jussieu.fr/~rousse/gpgkey.html

1 0

Re: is there an "xml-include"
by Gregor Hagedorn 20 Nov '01

20 Nov '01

> This is a means to separate out a schema into multiple files, > if I understand Gregor correctly he's asking for a way to test for > id/keyref uniqueness across multiple files? Yes, and data files, not schema/dtd. One xml data file defines ids, the other points to these ids. In the case of descriptive data, 1000s of item description files should all point to the same, centrally defined character definitions. > At a higher level there is XInclude which allows for including > content from one document into another. Not sure on > current support for this however. I think this comes closest. Can you point me to information on XInclude? As you say, it seems unclear whether a validating parser, say msxml 4.0 would be able to check such id/idrefs. Gregor ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: XML: is there an "xml-include"
by Gregor Hagedorn 20 Nov '01

20 Nov '01

> Yes. See > http://www.w3.org/TR/xmlschema-0/#SchemaInMultDocs I think this is not possible. If I understand correctly, you are referring to schema, not idref in one xml document referring to id defined in another document. The schema would define the fact that two attribute (say cid and cidref) have to be interpreted as key and keyref, but not the actual values. ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: is there an "xml-include"
by Barringer, Kerry 20 Nov '01

20 Nov '01

Gregor, We disagree and I'd like to get to the reason why because I am sure that both of us are not fully understanding what the other is saying. I agree that a character name is not necessarily unique across different taxa. A character is what a person defines it to be. But I don't understand what you mean when you say that the name cannot stand for the character, that only a code can. This seems absurd. I don't see the difference between my 'name' and your 'code', especially when your code is just a compact coding that refers to a name. In your haircolor example, I think you are misunderstanding the hierarcical nature of XML coding. Each character is part of another, which all eventually are part of a 'root,' in our case probably a plant. So the character 'Plant' has the characters 'stems.' 'leaves,' etc. 'Leaves" may contain the character 'trichomes' which may contain the character 'color.' If I understand what you wrote, your criticism applies to a relational model where the character of 'aircolor' might be used for leaf trichomes, fine enations inside a fruit, or the body covering of mammals. It is impossible to define a character this way in an XML file because it is strictly hierarcical, not relational. If you feel that descriptive data can only be modeled with relations, then you can never get XML to work for you. It is not relational, it is strictly hierarcical. It does not and, by design, cannot support the relational model. Do not confuse XML with a database. It is -not- designed to be a database. The few databases that use XML are all hierarcical XML is designed to model the structure of documents. If you want to create a database, use a database language. XML is what you use in the report. My feeling is that the hierarcical nature of XML might be suitable for plant descriptions. I know it does a good job coding taxomomic papers and taxonomic nomenclature. It is less suitable, though still workable, for specimen data, which is easier to model with relational tables. To use XML though, I think we must look outside the DELTA model of character coding. XML won't improve the DELTA model or fix its faults. We must also look beyond the database model. Plant descriptions are highly structured documents. XML can model that structure. Finally, we have to get beyond the wordiness of XML files. They are designed to be that way. we can only live with it. Let me know if this makes sense and be sure to correct me if you feel I am being too stupid. all the best, Kerry -----Original Message----- From: Gregor Hagedorn [mailto:G.Hagedorn@BBA.DE] Sent: Tuesday, November 20, 2001 10:31 AM To: TDWG-SDD(a)USOBI.ORG Subject: Re: is there an "xml-include" Dear Kerry, I can not see how what you propose could work. The name of a character is not necessarily unique across different taxa, and often not even within a taxon. Further the character concept and its names (technical, laymen, English, German, etc.) are a 1:n relation. The name can not stand for the character, only a code can. I do not care whether this code is characters or numbers, but I believe it is a mistake to think if something is <haircolor></haircolor> to take the code at face value and assume you know what are hairs and what is color. Hairs are quite different things in plants, animals, or fungi, and color needs information whether it is a code from a color comparison chart, or an undefined term like "red". I think I am sticking less with a DELTA storage optimization model, but with a relational information model, which is what I have been using for all my projects. The relational model allows language independence. How can you preserve that, without having unique codes that lead to the definition of a character? That does not mean, that a free text description in some language, say Chinese, may be present, in addition to the data. That is why I am thinking of attributes, not element data. Gregor ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: is there an "xml-include"
by Gregor Hagedorn 20 Nov '01

20 Nov '01

Dear Kerry, I can not see how what you propose could work. The name of a character is not necessarily unique across different taxa, and often not even within a taxon. Further the character concept and its names (technical, laymen, English, German, etc.) are a 1:n relation. The name can not stand for the character, only a code can. I do not care whether this code is characters or numbers, but I believe it is a mistake to think if something is <haircolor></haircolor> to take the code at face value and assume you know what are hairs and what is color. Hairs are quite different things in plants, animals, or fungi, and color needs information whether it is a code from a color comparison chart, or an undefined term like "red". I think I am sticking less with a DELTA storage optimization model, but with a relational information model, which is what I have been using for all my projects. The relational model allows language independence. How can you preserve that, without having unique codes that lead to the definition of a character? That does not mean, that a free text description in some language, say Chinese, may be present, in addition to the data. That is why I am thinking of attributes, not element data. Gregor ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: XML: is there an "xml-include"
by Bryan Heidorn 20 Nov '01

20 Nov '01

At 11:17 AM 11/20/01 +0100, you wrote: >If character are defined with an id (numeric or character) in one >file, and the item descriptions use these ids through idref or >through xml-schema keyref means: > >How can a validating parser validate the schema, including keyrefs, >without having to include the entire character definition in each of >1000s of taxon description xml files? Can the ids for idref/keyref be >declared to be in a separate file? The character definition for a class of taxa, including the list of valid states could be included as part of the character-schema. An "include" can be used to insert the character schema within a taxon schema. The character schema would allow the validator to enforce the limitations on any particular xml instance file. As pointed out by Kerry Barringer I do not believe that there is a need in XML to use an additional level of indirection to include the item definitions in another file. This will cost space but there is nothing to keep the XML from being stored in a relational database and constructed on the fly as needed. If we want to enforce relational normalization it might best be done in the underlying database not in the xml instantiation of a view of the data. It might be cleaner if the XML validator could do this but for the purposes of data interchange and even interoperability, I think it is still useful for the xml validator to be able to check the xml documents even without the relational constraints. What is lost is that if you are moving data from one database to another, a program might not be able to easily recognize data relations and the receiving database might inefficiently save the data in a different database structure from the originating database. Bryan Heidorn >I could not find a standard xml-include command. I know there is one >defined for dtds, but that is all I could find. > >Can the xml experts help? > >Gregor > >PS I will post minutes of the TDWG meeting within the next days >---------------------------------------------------------- >Gregor Hagedorn (G.Hagedorn(a)bba.de) >Institute for Plant Virology, Microbiology, and Biosafety >Federal Research Center for Agriculture and Forestry (BBA) >Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 >14195 Berlin, Germany Fax: +49-30-8304-2203 > >Often wrong but never in doubt! -- -------------------------------------------------------------------- P. Bryan Heidorn Graduate School of Library and Information Science pheidorn(a)uiuc.edu University of Illinois at Urbana-Champaign MC-493 (V)217/ 244-7792 Rm. 221, 501 East Daniel St., Champaign, IL 61820-6212 (F)217/ 244-3302 http://alexia.lis.uiuc.edu/~heidorn Calendar: http://calendar.yahoo.com/pbheidorn Visit the Biobrowser Web site at http://www.biobrowser.org

1 0

Re: is there an "xml-include"
by Leigh Dodds 20 Nov '01

20 Nov '01

This is a means to separate out a schema into multiple files, if I understand Gregor correctly he's asking for a way to test for id/keyref uniqueness across multiple files? If so, then you can include chunks of XML in other documents using the dtd entity mechanism -- which is what I believe he has already tried. I don't believe there's actually an equivalent for XML Schema: you still need a DTD. At a higher level there is XInclude which allows for including content from one document into another. Not sure on current support for this however. (Assuming I'm on the right track) You may want to consider using an additional validation mechanism here, such as XLinkIt [1] which can make assertions (e.g. id uniqueness) across multiple documents. Hope that helps, [1]. http://www.xlinkit.com/main.html > -----Original Message----- > From: TDWG - Structure of Descriptive Data [mailto:TDWG-SDD@usobi.org]On > Behalf Of Robert A. (Bob) Morris > Sent: 20 November 2001 11:41 > To: TDWG-SDD(a)usobi.org > Subject: XML: is there an "xml-include" > > > Yes. See > http://www.w3.org/TR/xmlschema-0/#SchemaInMultDocs > > > Gregor Hagedorn writes: > > Date: Tue, 20 Nov 2001 11:17:27 +0100 > > From: Gregor Hagedorn <G.Hagedorn(a)bba.de> > > To: TDWG-SDD(a)usobi.org > > Subject: XML: is there an "xml-include" > > > > If character are defined with an id (numeric or character) in one > > file, and the item descriptions use these ids through idref or > > through xml-schema keyref means: > > > > How can a validating parser validate the schema, including keyrefs, > > without having to include the entire character definition in each of > > 1000s of taxon description xml files? Can the ids for idref/keyref be > > declared to be in a separate file? > > > > I could not find a standard xml-include command. I know there is one > > defined for dtds, but that is all I could find. > > > > Can the xml experts help? > > > > Gregor > > > > PS I will post minutes of the TDWG meeting within the next days > > ---------------------------------------------------------- > > Gregor Hagedorn (G.Hagedorn(a)bba.de) > > Institute for Plant Virology, Microbiology, and Biosafety > > Federal Research Center for Agriculture and Forestry (BBA) > > Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 > > 14195 Berlin, Germany Fax: +49-30-8304-2203 > > > > Often wrong but never in doubt! > >

1 0

XML: is there an "xml-include"
by Gregor Hagedorn 20 Nov '01

20 Nov '01

If character are defined with an id (numeric or character) in one file, and the item descriptions use these ids through idref or through xml-schema keyref means: How can a validating parser validate the schema, including keyrefs, without having to include the entire character definition in each of 1000s of taxon description xml files? Can the ids for idref/keyref be declared to be in a separate file? I could not find a standard xml-include command. I know there is one defined for dtds, but that is all I could find. Can the xml experts help? Gregor PS I will post minutes of the TDWG meeting within the next days ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0

Re: is there an "xml-include"
by Robert A. (Bob) Morris 20 Nov '01

20 Nov '01

Ah, you are correct. I read his request too quickly. Both Apache Cocoon and Microsoft .NET claim to support XInclude but I am not sure if there are any pure client-side implementations. Also, it is unclear at (another too quick) glance what the validation issues may be with either of these. Leigh Dodds writes: > Date: Tue, 20 Nov 2001 12:29:42 -0000 > From: Leigh Dodds <ldodds(a)INGENTA.COM> > To: TDWG-SDD(a)usobi.org > Subject: Re: is there an "xml-include" > > This is a means to separate out a schema into multiple files, > if I understand Gregor correctly he's asking for a way to test for > id/keyref uniqueness across multiple files? > > If so, then you can include chunks of XML in other documents > using the dtd entity mechanism -- which is what I believe he > has already tried. I don't believe there's actually an equivalent > for XML Schema: you still need a DTD. > > At a higher level there is XInclude which allows for including > content from one document into another. Not sure on > current support for this however. > > (Assuming I'm on the right track) You may want to consider > using an additional validation mechanism here, such as XLinkIt [1] > which can make assertions (e.g. id uniqueness) across multiple > documents. > > Hope that helps, > > [1]. http://www.xlinkit.com/main.html > > > -----Original Message----- > > From: TDWG - Structure of Descriptive Data [mailto:TDWG-SDD@usobi.org]On > > Behalf Of Robert A. (Bob) Morris > > Sent: 20 November 2001 11:41 > > To: TDWG-SDD(a)usobi.org > > Subject: XML: is there an "xml-include" > > > > > > Yes. See > > http://www.w3.org/TR/xmlschema-0/#SchemaInMultDocs > > > > > > Gregor Hagedorn writes: > > > Date: Tue, 20 Nov 2001 11:17:27 +0100 > > > From: Gregor Hagedorn <G.Hagedorn(a)bba.de> > > > To: TDWG-SDD(a)usobi.org > > > Subject: XML: is there an "xml-include" > > > > > > If character are defined with an id (numeric or character) in one > > > file, and the item descriptions use these ids through idref or > > > through xml-schema keyref means: > > > > > > How can a validating parser validate the schema, including keyrefs, > > > without having to include the entire character definition in each of > > > 1000s of taxon description xml files? Can the ids for idref/keyref be > > > declared to be in a separate file? > > > > > > I could not find a standard xml-include command. I know there is one > > > defined for dtds, but that is all I could find. > > > > > > Can the xml experts help? > > > > > > Gregor > > > > > > PS I will post minutes of the TDWG meeting within the next days > > > ---------------------------------------------------------- > > > Gregor Hagedorn (G.Hagedorn(a)bba.de) > > > Institute for Plant Virology, Microbiology, and Biosafety > > > Federal Research Center for Agriculture and Forestry (BBA) > > > Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 > > > 14195 Berlin, Germany Fax: +49-30-8304-2203 > > > > > > Often wrong but never in doubt! > > > >

1 0

Re: is there an "xml-include"
by Barringer, Kerry 20 Nov '01

20 Nov '01

Gregor, My understanding of XML is that indeed, the character name and states should be written out fully in all descriptions. Characters and states should not be placed in a separate file aws in DELTA. The 'efficiency' of the DELTA model is really a trade off optimizing storage over processing. The makers of the XML model deliberately chose not to optimize storage and to create wordy XML files to optimize processing. File storage is not viewed as being a limiting factor any more, as it was when DELTA was designed. There are a few ways to include data from other files and DTD's but these are best used to create templates and allow for differently structured DTD in the same document. Say you have a DTD describing a taxonomic paper. That DTD can easily reference separate DTD's for taxonomic nomenclature, plants specimens, and taxon descriptions, combining the results which can then be transformed into a full taxonomic treatment. I think it is a mistake to try to directly translate the DELTA data model into XML. The XML model is different. It is more flexible, more readable, and more easily processed. It can also model hierarchies, though its relational capabilities are poor. The focus should be on developing a good XML model of taxonomic data. If necessary, it can be easily translated into DELTA format, or any other format, using XSL and can be processed by the available XML tools. with Best regards Kerry Barringer Brooklyn Botanic Garden -----Original Message----- From: Gregor Hagedorn [mailto:G.Hagedorn@BBA.DE] Sent: Tuesday, November 20, 2001 5:17 AM To: TDWG-SDD(a)USOBI.ORG Subject: XML: is there an "xml-include" If character are defined with an id (numeric or character) in one file, and the item descriptions use these ids through idref or through xml-schema keyref means: How can a validating parser validate the schema, including keyrefs, without having to include the entire character definition in each of 1000s of taxon description xml files? Can the ids for idref/keyref be declared to be in a separate file? I could not find a standard xml-include command. I know there is one defined for dtds, but that is all I could find. Can the xml experts help? Gregor PS I will post minutes of the TDWG meeting within the next days ---------------------------------------------------------- Gregor Hagedorn (G.Hagedorn(a)bba.de) Institute for Plant Virology, Microbiology, and Biosafety Federal Research Center for Agriculture and Forestry (BBA) Koenigin-Luise-Str. 19 Tel: +49-30-8304-2220 14195 Berlin, Germany Fax: +49-30-8304-2203 Often wrong but never in doubt!

1 0