January 2006 - tdwg-tag - lists.tdwg.org

[tdwg-tapir] Schema modification: r393 - trunk/protocol
by tdwg-tapir＠lists.tdwg.org 05 Jan '06

05 Jan '06

Author: renato Date: 2006-01-05 18:53:02 +0100 (Thu, 05 Jan 2006) New Revision: 393 Modified: trunk/protocol/tapir.xsd Log: * Reduced "queryTemplateGroup" to contain only <label> and <documentation>. * Moved the other elements from "queryTemplateGroup" to a new "searchTemplateGroup" (which also references the "queryTemplateGroup"). * Created a new "inventoryTemplateGroup" in the same way, and made the inventory request element reference it (as it happens in the search request definition). * Removed the global element <queryTemplate> and included <searchTemplate> and <inventoryTemplate> to make the schema validate external template definitions. This way query templates used in view operations can point to predefined search or inventory operations. * Removed the possibility to use queryTemplateURI in XML search requests. Modified: trunk/protocol/tapir.xsd =================================================================== --- trunk/protocol/tapir.xsd 2006-01-05 17:32:48 UTC (rev 392) +++ trunk/protocol/tapir.xsd 2006-01-05 17:53:02 UTC (rev 393) @@ -254,15 +254,20 @@ </xsd:element> </xsd:sequence> </xsd:group> - <xsd:element name="queryTemplate"> + <xsd:group name="queryTemplateGroup"> + <xsd:sequence> + <xsd:element name="label" type="xsd:string" minOccurs="0"/> + <xsd:element name="documentation" type="xsd:string" minOccurs="0"/> + </xsd:sequence> + </xsd:group> + <xsd:element name="searchTemplate"> <xsd:complexType> - <xsd:group ref="queryTemplateGroup"/> + <xsd:group ref="searchTemplateGroup"/> </xsd:complexType> </xsd:element> - <xsd:group name="queryTemplateGroup"> + <xsd:group name="searchTemplateGroup"> <xsd:sequence> - <xsd:element name="label" type="xsd:string" minOccurs="0"/> - <xsd:element name="documentation" type="xsd:string" minOccurs="0"/> + <xsd:group ref="queryTemplateGroup"/> <xsd:element name="outputModelURI"> <xsd:annotation> <xsd:documentation>A simple XPath pointing to a response structure node that @@ -304,6 +309,33 @@ </xsd:element> </xsd:sequence> </xsd:group> + <xsd:element name="inventoryTemplate"> + <xsd:complexType> + <xsd:group ref="inventoryTemplateGroup"/> + </xsd:complexType> + </xsd:element> + <xsd:group name="inventoryTemplateGroup"> + <xsd:sequence> + <xsd:group ref="queryTemplateGroup"/> + <xsd:element name="concepts"> + <xsd:annotation> + <xsd:documentation>Used to specify a list of concepts to + make an inventory on.</xsd:documentation> + </xsd:annotation> + <xsd:complexType> + <xsd:sequence> + <xsd:element name="concept" type="qualifiedConceptReference" minOccurs="0" maxOccurs="unbounded"> + <xsd:annotation> + <xsd:documentation>A single qualified + concept</xsd:documentation> + </xsd:annotation> + </xsd:element> + </xsd:sequence> + </xsd:complexType> + </xsd:element> + <xsd:element ref="filter" minOccurs="0"/> + </xsd:sequence> + </xsd:group>    @@ -636,23 +668,7 @@ <xsd:element name="inventory"> <xsd:complexType> <xsd:sequence> - <xsd:element name="concepts"> - <xsd:annotation> - <xsd:documentation>Used to specify a list of concepts to - make an inventory on.</xsd:documentation> - </xsd:annotation> - <xsd:complexType> - <xsd:sequence> - <xsd:element name="concept" type="qualifiedConceptReference" minOccurs="0" maxOccurs="unbounded"> - <xsd:annotation> - <xsd:documentation>A single qualified - concept</xsd:documentation> - </xsd:annotation> - </xsd:element> - </xsd:sequence> - </xsd:complexType> - </xsd:element> - <xsd:element ref="filter" minOccurs="0"/> + <xsd:group ref="inventoryTemplateGroup"/> </xsd:sequence> <xsd:attributeGroup ref="operationRequestGroup"/> <xsd:attributeGroup ref="pagingParameters"/> @@ -663,21 +679,11 @@ <xsd:documentation>A search with the view definition used for the search as an attribute. If left out defaults to the one specified by the provider and exposed in the capabilities.</xsd:documentation> </xsd:annotation> <xsd:complexType> - <xsd:choice> - <xsd:element name="queryTemplateURI"> - <xsd:complexType> - <xsd:complexContent> - <xsd:extension base="externalResourceType"> - <xsd:attribute name="reload" type="xsd:boolean" use="optional" default="false"/> - </xsd:extension> - </xsd:complexContent> - </xsd:complexType> - </xsd:element> - <xsd:sequence> - <xsd:group ref="queryTemplateGroup"/> - <xsd:element name="partial" minOccurs="0"> - <xsd:annotation> - <xsd:documentation>Select only a subset of the + <xsd:sequence> + <xsd:group ref="searchTemplateGroup"/> + <xsd:element name="partial" minOccurs="0"> + <xsd:annotation> + <xsd:documentation>Select only a subset of the entire response structure to be used in the response. Useful for working with large cached external structures referenced via URLs. Select @@ -688,21 +694,20 @@ structure, which are not listed in the partial search ,must be included in the response content.</xsd:documentation> - </xsd:annotation> - <xsd:complexType> - <xsd:sequence> - <xsd:element name="node" type="nodeReference" maxOccurs="unbounded"> - <xsd:annotation> - <xsd:documentation>A node taken from the + </xsd:annotation> + <xsd:complexType> + <xsd:sequence> + <xsd:element name="node" type="nodeReference" maxOccurs="unbounded"> + <xsd:annotation> + <xsd:documentation>A node taken from the response structure definition.</xsd:documentation> - </xsd:annotation> - </xsd:element> - </xsd:sequence> - </xsd:complexType> - </xsd:element> - </xsd:sequence> - </xsd:choice> + </xsd:annotation> + </xsd:element> + </xsd:sequence> + </xsd:complexType> + </xsd:element> + </xsd:sequence> <xsd:attributeGroup ref="operationRequestGroup"/> <xsd:attributeGroup ref="pagingParameters"/> </xsd:complexType>

1 0

[tdwg-tapir] Schema modification: r392 - trunk/protocol
by tdwg-tapir＠lists.tdwg.org 05 Jan '06

05 Jan '06

Author: renato Date: 2006-01-05 18:32:48 +0100 (Thu, 05 Jan 2006) New Revision: 392 Modified: trunk/protocol/tapir.xsd Log: * Reincluded the "view" operation which will be completely dedicated to the usage of queryTemples. This operation will only be accessible through GET whithout using XML requests, and it will not accept the "filter" and "partial" parameters. * Moved /response/capabilities/operations/search/queryTemplates to /response/capabilities/operations/view/queryTemplates. * Included a new <predefinedOutputModels> element in /response/capabilities/operations/search. Modified: trunk/protocol/tapir.xsd =================================================================== --- trunk/protocol/tapir.xsd 2006-01-05 17:19:49 UTC (rev 391) +++ trunk/protocol/tapir.xsd 2006-01-05 17:32:48 UTC (rev 392) @@ -787,18 +787,30 @@ <xsd:element name="capabilities"> <xsd:complexType/> </xsd:element> + <xsd:element name="view" minOccurs="0"> + <xsd:complexType> + <xsd:sequence> + <xsd:element name="queryTemplates"> + <xsd:complexType> + <xsd:sequence> + <xsd:element name="queryTemplate" type="externalResourceType" maxOccurs="unbounded"/> + </xsd:sequence> + </xsd:complexType> + </xsd:element> + </xsd:sequence> + </xsd:complexType> + </xsd:element> <xsd:element name="inventory" minOccurs="0"> <xsd:complexType/> </xsd:element> <xsd:element name="search" minOccurs="0"> <xsd:complexType> <xsd:sequence> - <xsd:element name="queryTemplates"> + <xsd:element name="predefinedOutputModels"> <xsd:complexType> <xsd:sequence> - <xsd:element name="queryTemplate" type="externalResourceType" minOccurs="0" maxOccurs="unbounded"/> + <xsd:element name="outputModel" type="externalResourceType" minOccurs="0" maxOccurs="unbounded"/> </xsd:sequence> - <xsd:attribute name="default" type="xsd:string" use="optional"/> </xsd:complexType> </xsd:element> <xsd:element name="customOutputModels"> @@ -815,111 +827,6 @@ <xsd:attribute name="accepted" type="xsd:boolean" use="required"/> </xsd:complexType> </xsd:element> - <xsd:element name="variables"> - <xsd:complexType> - <xsd:sequence> - <xsd:element name="environment" minOccurs="0"> - <xsd:complexType> - <xsd:sequence minOccurs="0"> - <xsd:element name="accessPoint" minOccurs="0"> - <xsd:annotation> - <xsd:documentation>An element to - represent the access point - of the datasource. - Substitutes an abstract - environment expression - element.</xsd:documentation> - </xsd:annotation> - <xsd:complexType/> - </xsd:element> - <xsd:element name="datasourceName" minOccurs="0"> - <xsd:annotation> - <xsd:documentation>An element to - represent the name of the - datasource. Substitutes an - abstract environment - expression - element.</xsd:documentation> - </xsd:annotation> - <xsd:complexType/> - </xsd:element> - <xsd:element name="lastUpdate" minOccurs="0"> - <xsd:annotation> - <xsd:documentation>An element to - represent the timestamp of - the last update of the - datasource. Substitutes an - abstract environment - expression - element.</xsd:documentation> - </xsd:annotation> - <xsd:complexType/> - </xsd:element> - <xsd:element name="date" minOccurs="0"> - <xsd:annotation> - <xsd:documentation>An element to - represent ISO date like - 2005-07-21. Substitutes an - abstract environment - expression - element.</xsd:documentation> - </xsd:annotation> - <xsd:complexType/> - </xsd:element> - <xsd:element name="timestamp" minOccurs="0"> - <xsd:annotation> - <xsd:documentation>An element to - represent an ISO timestamp - like 2005-07-20 17:01:38. - Substitutes an abstract - environment expression - element.</xsd:documentation> - </xsd:annotation> - <xsd:complexType/> - </xsd:element> - <xsd:element name="metadata" minOccurs="0"> - <xsd:annotation> - <xsd:documentation>Variables - taken from the metadata - response - definition.</xsd:documentation> - </xsd:annotation> - <xsd:complexType/> - </xsd:element> - <xsd:element name="custom" minOccurs="0"> - <xsd:annotation> - <xsd:documentation>Information - about the local - configuration of a - datasource.</xsd:documentation> - </xsd:annotation> - <xsd:complexType> - <xsd:sequence> - <xsd:any minOccurs="0" maxOccurs="unbounded"> - <xsd:annotation> - <xsd:documentation>For - any custom - extension. E.g - what security - related - capabilities - does the - provider have? - Encryption, - access - restriction, - etc.</xsd:documentation> - </xsd:annotation> - </xsd:any> - </xsd:sequence> - </xsd:complexType> - </xsd:element> - </xsd:sequence> - </xsd:complexType> - </xsd:element> - </xsd:sequence> - </xsd:complexType> - </xsd:element> </xsd:sequence> </xsd:complexType> </xsd:element>

1 0

[tdwg-tapir] Schema modification: r391 - trunk/protocol
by tdwg-tapir＠lists.tdwg.org 05 Jan '06

05 Jan '06

Author: renato Date: 2006-01-05 18:19:49 +0100 (Thu, 05 Jan 2006) New Revision: 391 Modified: trunk/protocol/tapir.xsd Log: Changed multiCOPType (basis of the <in> operator) so that <values> can contain a sequence of "expression" elements instead of only literals. This will allow parameters with multiple values (or multiple parameters) or even concepts. Modified: trunk/protocol/tapir.xsd =================================================================== --- trunk/protocol/tapir.xsd 2006-01-05 16:50:35 UTC (rev 390) +++ trunk/protocol/tapir.xsd 2006-01-05 17:19:49 UTC (rev 391) @@ -491,7 +491,7 @@ <xsd:element name="values"> <xsd:complexType> <xsd:sequence> - <xsd:element ref="literal" maxOccurs="unbounded"/> + <xsd:element ref="expression" maxOccurs="unbounded"/> </xsd:sequence> </xsd:complexType> </xsd:element>

1 0

[tdwg-tapir] Schema modification: r390 - trunk/protocol
by tdwg-tapir＠lists.tdwg.org 05 Jan '06

05 Jan '06

Author: renato Date: 2006-01-05 17:50:35 +0100 (Thu, 05 Jan 2006) New Revision: 390 Modified: trunk/protocol/tapir.xsd Log: Changed filter capabilities so that implementations either accept all filter functionality or none: * Renamed element /response/capabilities/operators to "filter" and included an optional subelement called "encoding". * Renamed "boolean" to "booleanOperators", and made all logical and comparative operators mandatory. Now parsers can always rely on the existence of a <filter> element in the capabilities response and then pass it to a specific handler (such handler can just detect if there's an element "encoding" or not, which means "I understand the basic filtering mechanism" or "I don't know how to parse any filters"). Modified: trunk/protocol/tapir.xsd =================================================================== --- trunk/protocol/tapir.xsd 2006-01-05 14:51:36 UTC (rev 389) +++ trunk/protocol/tapir.xsd 2006-01-05 16:50:35 UTC (rev 390) @@ -2,7 +2,7 @@    -<xsd:schema xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" xmlns:xsd="http://www.w3.org/2001/XMLSchema" targetNamespace="http://www.tdwg.org/schemas/tapir/datasource/1.0" elementFormDefault="qualified" attributeFormDefault="unqualified" version="1.0" xml:lang="EN"> +<xsd:schema targetNamespace="http://www.tdwg.org/schemas/tapir/datasource/1.0" elementFormDefault="qualified" attributeFormDefault="unqualified" version="1.0" xml:lang="EN" xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" xmlns:xsd="http://www.w3.org/2001/XMLSchema">    @@ -926,7 +926,7 @@ </xsd:sequence> </xsd:complexType> </xsd:element> - <xsd:element name="operators" type="filterCapabilities"> + <xsd:element name="filter" type="filterCapabilities"> <xsd:annotation> <xsd:documentation>Information about the supported filter operators of this service.</xsd:documentation> @@ -1336,115 +1336,121 @@ service.</xsd:documentation> </xsd:annotation> <xsd:sequence> - <xsd:element name="expression"> + <xsd:element name="encoding" minOccurs="0"> <xsd:complexType> <xsd:sequence> - <xsd:element name="concept"> - <xsd:complexType/> - </xsd:element> - <xsd:element name="literal"> - <xsd:complexType/> - </xsd:element> - <xsd:element name="parameter"> - <xsd:complexType/> - </xsd:element> - <xsd:element name="variable"> - <xsd:complexType/> - </xsd:element> - <xsd:element name="arithmetic"> - <xsd:annotation> - <xsd:documentation>Support +-*/ in filter - expressions.</xsd:documentation> - </xsd:annotation> + <xsd:element name="expression"> <xsd:complexType> <xsd:sequence> - <xsd:element name="add" minOccurs="0"> + <xsd:element name="concept"> <xsd:complexType/> </xsd:element> - <xsd:element name="sub" minOccurs="0"> + <xsd:element name="literal"> <xsd:complexType/> </xsd:element> - <xsd:element name="div" minOccurs="0"> + <xsd:element name="parameter"> <xsd:complexType/> </xsd:element> - <xsd:element name="mul" minOccurs="0"> + <xsd:element name="variable"> <xsd:complexType/> </xsd:element> - <xsd:element ref="custom" minOccurs="0"/> - </xsd:sequence> - </xsd:complexType> - </xsd:element> - <xsd:element ref="custom" minOccurs="0"/> - </xsd:sequence> - </xsd:complexType> - </xsd:element> - <xsd:element name="boolean"> - <xsd:complexType> - <xsd:sequence> - <xsd:element name="logical"> - <xsd:annotation> - <xsd:documentation>This includes all 3 operators. AND, OR, - NOT</xsd:documentation> - </xsd:annotation> - <xsd:complexType> - <xsd:sequence> - <xsd:element name="not" minOccurs="0"> - <xsd:complexType/> + <xsd:element name="arithmetic"> + <xsd:annotation> + <xsd:documentation>Support +-*/ in filter + expressions.</xsd:documentation> + </xsd:annotation> + <xsd:complexType> + <xsd:sequence> + <xsd:element name="add" minOccurs="0"> + <xsd:complexType/> + </xsd:element> + <xsd:element name="sub" minOccurs="0"> + <xsd:complexType/> + </xsd:element> + <xsd:element name="div" minOccurs="0"> + <xsd:complexType/> + </xsd:element> + <xsd:element name="mul" minOccurs="0"> + <xsd:complexType/> + </xsd:element> + <xsd:element ref="custom" minOccurs="0"/> + </xsd:sequence> + </xsd:complexType> </xsd:element> - <xsd:element name="and" minOccurs="0"> - <xsd:complexType/> - </xsd:element> - <xsd:element name="or" minOccurs="0"> - <xsd:complexType/> - </xsd:element> <xsd:element ref="custom" minOccurs="0"/> </xsd:sequence> </xsd:complexType> </xsd:element> - <xsd:element name="comparative"> + <xsd:element name="booleanOperators"> <xsd:complexType> <xsd:sequence> - <xsd:element name="equals" minOccurs="0"> + <xsd:element name="logical"> <xsd:annotation> - <xsd:documentation>Recommended to be case - sensitive.</xsd:documentation> + <xsd:documentation>This includes all 3 operators. AND, OR, + NOT</xsd:documentation> </xsd:annotation> <xsd:complexType> - <xsd:attribute name="caseSensitive" type="xsd:boolean" use="optional" default="true"/> + <xsd:sequence> + <xsd:element name="not"> + <xsd:complexType/> + </xsd:element> + <xsd:element name="and"> + <xsd:complexType/> + </xsd:element> + <xsd:element name="or"> + <xsd:complexType/> + </xsd:element> + <xsd:element ref="custom" minOccurs="0"/> + </xsd:sequence> </xsd:complexType> </xsd:element> - <xsd:element name="greaterThan" minOccurs="0"> - <xsd:complexType/> - </xsd:element> - <xsd:element name="greaterThanOrEquals" minOccurs="0"> - <xsd:complexType/> - </xsd:element> - <xsd:element name="lessThan" minOccurs="0"> - <xsd:complexType/> - </xsd:element> - <xsd:element name="lessThanOrEquals" minOccurs="0"> - <xsd:complexType/> - </xsd:element> - <xsd:element name="in" minOccurs="0"> - <xsd:complexType/> - </xsd:element> - <xsd:element name="isNull" minOccurs="0"> - <xsd:complexType/> - </xsd:element> - <xsd:element name="like" minOccurs="0"> - <xsd:annotation> - <xsd:documentation>Recommended to be case insensitive. + <xsd:element name="comparative"> + <xsd:complexType> + <xsd:sequence> + <xsd:element name="equals"> + <xsd:annotation> + <xsd:documentation>Recommended to be case + sensitive.</xsd:documentation> + </xsd:annotation> + <xsd:complexType> + <xsd:attribute name="caseSensitive" type="xsd:boolean" use="optional" default="true"/> + </xsd:complexType> + </xsd:element> + <xsd:element name="greaterThan"> + <xsd:complexType/> + </xsd:element> + <xsd:element name="greaterThanOrEquals"> + <xsd:complexType/> + </xsd:element> + <xsd:element name="lessThan"> + <xsd:complexType/> + </xsd:element> + <xsd:element name="lessThanOrEquals"> + <xsd:complexType/> + </xsd:element> + <xsd:element name="in"> + <xsd:complexType/> + </xsd:element> + <xsd:element name="isNull"> + <xsd:complexType/> + </xsd:element> + <xsd:element name="like"> + <xsd:annotation> + <xsd:documentation>Recommended to be case insensitive. Wildcard is always asterisk *.</xsd:documentation> - </xsd:annotation> - <xsd:complexType> - <xsd:attribute name="caseSensitive" type="xsd:boolean" use="optional" default="false"/> + </xsd:annotation> + <xsd:complexType> + <xsd:attribute name="caseSensitive" type="xsd:boolean" use="optional" default="false"/> + </xsd:complexType> + </xsd:element> + <xsd:element ref="custom" minOccurs="0"/> + </xsd:sequence> </xsd:complexType> </xsd:element> <xsd:element ref="custom" minOccurs="0"/> </xsd:sequence> </xsd:complexType> </xsd:element> - <xsd:element ref="custom" minOccurs="0"/> </xsd:sequence> </xsd:complexType> </xsd:element>

1 0

[tdwg-tapir] Proposal
by Renato De Giovanni 05 Jan '06

05 Jan '06

Dear all, I would like to propose a set of changes to the protocol trying to address the main issues that we've recently discussed here. The changes are described below and if you want to browse the corresponding schema you can find it here: http://www.cria.org.br/~renato/tapir.xsd The main issues are related to TapirLite (how to accomodate it without making generic clients overly complicated), and are also related to Donald's suggestion about inventory templates. All other changes proposed were a mere consequence of browsing the schema and finding parts that I thought could be improved. Please, take a look when you have some time and let me know if you have any comments, suggestions, objections, etc. Personally I'm happy with the changes, so if you all agree, and if we don't find other issues, I'll suggest to "feature freeze" it as soon as possible so that people can (re)start implementation based on it. The feature freeze was originally planned for yesterday (December 15th), but since the current version still has open issues and this proposal should obviously be reviewed and discussed, I'll suggest that we postpone it for at least January 5th. By the way, I'll be on vacation during the next two weeks, so only accessing e-mail sporadically. Best Regards, -- Renato ------- TAPIR / proposed changes ------- * Change filter capabilities so that implementations either accept all filter functionality or none. Specific changes: rename element /response/capabilities/operators to "filter" and include an optional subelement called "encoding". Rename "boolean" to "booleanOperators", and make all logical and comparative operators mandatory. This way parsers can always rely on the existence of a <filter> element and then pass it to a specific handler (same technique suggested by Steve in other parts of the protocol). Such handler can just detect if there's an element "encoding" or not, which means "I understand the basic filtering mechanism" or "I don't know how to parse any filters". TapirLite implementations will answer in the capabilities: ... <filters/> ... Full TAPIR implementations will answer: ... <filters> <encoding> <expression> <concept/> <literal/> <parameter/> <variable/> <arithmetic/> </expression> <booleanOperators> <logical> <not/> <and/> <or/> </logical> <comparative> <equals/> <greaterThan/> <greaterThanOrEquals/> <lessThan/> <lessThanOrEquals/> <in/> <isNull/> <like/> </comparative> </booleanOperators> </encoding> </filters> ... * Change multiCOPType (basis of the <in> operator) so that <values> can contain a sequence of "expression" elements instead of only literals. This will allow parameters with multiple values (or multiple parameters) to be used by the TCS API. * Re-include an operation completely dedicated to the usage of queryTemples (I called it "view" but I'm happy with any other better names if people feel uncomfortable with it). This operation would only be accessible through GET whithout using XML requests. And it would not accept the "filter" and "partial" parameters. This seems to be the cleanest way to accomodate TapirLite since the _only_ change in the schema is the new optional element /response/capabilities/operations/view indicating the query templates supported. Obs: I tried other ways, but the results were not so nice and the changes were more drastic. * Remove the possibility to use queryTemplateURI in XML search requests. If someone wants to use a queryTemplateURI in a search request, it's so much simpler to use regular GET parameters in the URL that it's looks reasonable to simplify the protocol in these situations. * Move /response/capabilities/operations/search/queryTemplates to /response/capabilities/operations/view/queryTemplates. * On the other hand, include a new <predefinedOutputModels> element in /response/capabilities/operations/search (issue raised in a recent discussion with Rob in this mailing list). DiGIR2 will make use of it. * Reduce "queryTemplateGroup" to contain only <label> and <documentation>. Move the other elements to a new "searchTemplateGroup" which also references the "queryTemplateGroup". Create a new "inventoryTemplateGroup" in the same way, and make the inventory request element reference it (as it happens in the search request definition). Remove global element <queryTemplate>, including <searchTemplate> and <inventoryTemplate> to make the schema validate external template definitions. This way query templates used in view operations can point to predefined search or inventory operations. Other changes: * Make /response/capabilities/concepts/schema a mandatory element so that any implementation shows which concepts are present. * Remove controlled vocabulary of element /response/metadata/relatedEntities/entity/contact/role and make it unbounded: controlled vocabularies do help with standardization, but they also reduce flexibility. In different contexts people could easily envision different uses for that attribute, so I would rather enforce vocabulary control in other levels (such as software configuration or network configuration). * Remove "datum" attribute from geoLocationType: the corresponding element has been included so that "related entities" (part of metadata responses) could indicate their location. People can easily always put lat/long in WGS84 in this case, it's just a matter of adding a note about that in the documentation. * Rename primaryConceptNameServer and secondaryConceptNameServer to primary and secondary, respectively (the enclosing element already provides the context). Make <primary> optional. * Rename global element <environment> to <variable>, and move /response/capabilities/search/variables to /response/capabilities/variables since they can be used in both mappings and filters. Remove redefinition of all variables in the capabilities response by referencing the existing variable element defintion. Remove empty /response/capabilities/search/variables/environment/metadata element. -------

2 4

Re: [tdwg-tapir] reload / expire external resources
by "Döring, Markus" 05 Jan '06

05 Jan '06

I think I agree, Donald. We will probably just end up using the reload in all requests. If there are no further objections I am going to remove the attribute. BTW, the examples are updated and do validate against a slight modification of renatos current schema (but the examples still point to the SVN trunk which is outdated): http://ww2.biocase.org/svn/tapir/trunk/protocol/examples/ Markus -----Ursprüngliche Nachricht----- Von: Donald Hobern [mailto:dhobern@gbif.org] Gesendet: Donnerstag, 5. Januar 2006 15:39 An: Döring, Markus; tdwg-tapir(a)lists.tdwg.org Betreff: RE: [tdwg-tapir] reload / expire external resources It certainly makes sense, but I would expect our external documents to be static and relatively rarely changed, so it may be hard for resource providers to use the expiry value sensibly. The availability of the reload option in the request documents may also just encourage it to get used on every request (particularly since our requests are almost never hand-authored). Might it just be simpler to drop these attributes and leave it to provider developers to supply configuration options on how long they should cache external documents? I know that going with b) and c) is much more proper, but I'm not sure that it is worth it in our context. Donald --------------------------------------------------------------- Donald Hobern (dhobern(a)gbif.org) Programme Officer for Data Access and Database Interoperability Global Biodiversity Information Facility Secretariat Universitetsparken 15, DK-2100 Copenhagen, Denmark Tel: +45-35321483 Mobile: +45-28751483 Fax: +45-35321480 --------------------------------------------------------------- -----Original Message----- From: tdwg-tapir-bounces(a)lists.tdwg.org [mailto:tdwg-tapir-bounces@lists.tdwg.org] On Behalf Of "Döring, Markus" Sent: 05 January 2006 15:12 To: tdwg-tapir(a)lists.tdwg.org Subject: [tdwg-tapir] reload / expire external resources Hello everyone, I am in the process of scanning the schema for errors and update our examples in the SVN. Currently there's a reload attribute in the searchTemplateGroup used by the inline search operation & the template definition. It doesnt really make sense for the template document, does it? How should we handle this in general, maybe for all external resources? a) dont do anything about caching. remove reload/expire attributes b) allow external documents to specify an expiry date (like http) c) add optional reload attribute to all references to external docs In my implementation it was quite important to cache external documents that 1) need to be parsed/processed, like the xml schema docs and 2) that are not accessible all the time. So I think I am in favor of b) and c) actually. The requestor can always specify a reload and the external resource provider can specify a likely expiry date. Does this make sense or just complicates things? Markus _______________________________________________ tdwg-tapir mailing list tdwg-tapir(a)lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-tapir_lists.tdwg.org

1 0

[tdwg-tapir] Schema modification: r389 - trunk/protocol/examples
by tdwg-tapir＠lists.tdwg.org 05 Jan '06

05 Jan '06

Author: markus Date: 2006-01-05 15:51:36 +0100 (Thu, 05 Jan 2006) New Revision: 389 Added: trunk/protocol/examples/inventory_template1.xml trunk/protocol/examples/output_model1.xml trunk/protocol/examples/search_template1.xml Modified: trunk/protocol/examples/capabilities_request.xml trunk/protocol/examples/capabilities_response.xml trunk/protocol/examples/inventory_request.xml trunk/protocol/examples/inventory_response.xml trunk/protocol/examples/metadata_request.xml trunk/protocol/examples/metadata_response.xml trunk/protocol/examples/ping_request.xml trunk/protocol/examples/ping_response.xml Log: updated the examples to validate against renatos pre-christmas tapir schema and added examples for an output model document as well as search- and inventory templates. Modified: trunk/protocol/examples/capabilities_request.xml =================================================================== --- trunk/protocol/examples/capabilities_request.xml 2005-12-16 16:31:54 UTC (rev 388) +++ trunk/protocol/examples/capabilities_request.xml 2006-01-05 14:51:36 UTC (rev 389) @@ -1,10 +1,10 @@ -<?xml version="1.0" encoding="UTF-8"?> -<request xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" - xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> - <header> - <source accesspoint="http://url_from_client" sendtime="2005-11-11T12:23:56.023+01:00"> - </source> - </header> - <capabilities/> +<?xml version="1.0" encoding="UTF-8"?> +<request xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> + <header> + <source accesspoint="http://url_from_client" sendtime="2005-11-11T12:23:56.023+01:00"> + </source> + </header> + <capabilities/> </request> \ No newline at end of file Modified: trunk/protocol/examples/capabilities_response.xml =================================================================== --- trunk/protocol/examples/capabilities_response.xml 2005-12-16 16:31:54 UTC (rev 388) +++ trunk/protocol/examples/capabilities_response.xml 2006-01-05 14:51:36 UTC (rev 389) @@ -1,96 +1,102 @@ -<?xml version="1.0" encoding="UTF-8"?> -<response xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" - xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 - http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> - <header> - <source accesspoint="http://accesspoint_url" sendtime="2005-11-11T12:23:56.023+01:00"> - <software name="TAPIR_Implementation" version="1.0"/> - </source> - </header> - <capabilities> - <operations> - <ping/> - <metadata/> - <capabilities/> - <search> - <queryTemplates> - <queryTemplate location="http://url_to_querytemplate"/> - </queryTemplates> - <customOutputModels accepted="false"/> - <variables> - <environment> - <lastUpdate/> - <timestamp/> - <datasourceName/> - <date/> - <metadata/> - <accessPoint/> - <custom/> - </environment> - </variables> - </search> - </operations> - <operators> - <expression> - <concept/> - <literal/> - <parameter/> - <variable/> - <arithmetic> - <div/> - <mul/> - <add/> - <custom/> - </arithmetic> - </expression> - <boolean> - <logical> - <or/> - <not/> - <custom/> - <and/> - <custom/> - </logical> - <comparative> - <like/> - <lessThanOrEquals/> - <lessThan/> - <isNull/> - <in/> - <greaterThanOrEquals/> - <greaterThan/> - <equals/> - <custom/> - </comparative> - </boolean> - </operators> - <concepts> - <conceptNameServers> - <primaryConceptNameServer location="http://url_to_primary_concept_server"/> - <secondaryConceptNameServer location="http://url_to_secondary_concept_server"/> - </conceptNameServers> - <schema namespace="http://www.tdwg.org/schemas/abcd/1.2" - location="http://www.bgbm.org/TDWG/CODATA/Schema/ABCD-1.20.xsd"> - <mappedConcept - id="/DataSets/DataSet/DatasetDerivations/DatasetDerivation/DateSupplied" - searchable="true"/> - <mappedConcept - id="/DataSets/DataSet/DatasetDerivations/DatasetDerivation/Description" - searchable="true"/> - </schema> - <schema namespace="http://digir.net/schema/conceptual/darwin/extension/curatorial/1.0" - location="http://digir.net/schema/conceptual/darwin/extension/curatorial/1.0/curatori…"> - <mappedConcept id="/CatalogNumberNumeric" searchable="true"/> - <mappedConcept id="/IdentifiedBy" searchable="true"/> - </schema> - </concepts> - <settings> - <minQueryTermLength>2</minQueryTermLength> - <maxElementLevels>20</maxElementLevels> - <maxResponseSize>500</maxResponseSize> - <maxElementRepetitions>10</maxElementRepetitions> - <maxResponseTags>500</maxResponseTags> - </settings> - </capabilities> -</response> +<?xml version="1.0" encoding="UTF-8"?> +<response xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> + <header> + <source accesspoint="http://accesspoint_url" sendtime="2005-11-11T12:23:56.023+01:00"> + <software name="TAPIR_Implementation" version="1.0"/> + </source> + </header> + <capabilities> + <operations> + <ping/> + <metadata/> + <capabilities/> + <view> + <queryTemplates> + <queryTemplate location="http://url_to_searchtemplateA"/> + <queryTemplate location="http://url_to_searchtemplateB"/> + <queryTemplate location="http://url_to_inventorytemplate"/> + </queryTemplates> + </view> + <inventory/> + <search> + <predefinedOutputModels> + <outputModel location="http://url_to_raw_outputmodelA"/> + <outputModel location="http://url_to_raw_outputmodelB"/> + </predefinedOutputModels> + <customOutputModels accepted="false"/> + </search> + </operations> + <filter> + <encoding> + <expression> + <concept/> + <literal/> + <parameter/> + <variable/> + <arithmetic> + <add/> + <sub/> + </arithmetic> + </expression> + <booleanOperators> + <logical> + <not/> + <and/> + <or/> + </logical> + <comparative> + <equals/> + <greaterThan/> + <greaterThanOrEquals/> + <lessThan/> + <lessThanOrEquals/> + <in/> + <isNull/> + <like/> + </comparative> + </booleanOperators> + </encoding> + </filter> + <concepts> + <conceptNameServers> + <primary location="http://url_to_primary_concept_server"/> + <secondary location="http://url_to_secondary_concept_server"/> + </conceptNameServers> + <schema namespace="http://www.tdwg.org/schemas/abcd/1.2" + location="http://www.bgbm.org/TDWG/CODATA/Schema/ABCD-1.20.xsd"> + <mappedConcept + id="/DataSets/DataSet/DatasetDerivations/DatasetDerivation/DateSupplied" + searchable="true"/> + <mappedConcept + id="/DataSets/DataSet/DatasetDerivations/DatasetDerivation/Description" + searchable="true"/> + </schema> + <schema namespace="http://digir.net/schema/conceptual/darwin/extension/curatorial/1.0" + location="http://digir.net/schema/conceptual/darwin/extension/curatorial/1.0/curatori…"> + <mappedConcept id="/CatalogNumberNumeric" searchable="true" mandatory="true"/> + <mappedConcept id="/IdentifiedBy" searchable="true"/> + </schema> + </concepts> + + <variables> + <environment> + <timestamp/> + <lastUpdate/> + <date/> + <datasourceName/> + <accessPoint/> + </environment> + </variables> + + <settings> + <minQueryTermLength>2</minQueryTermLength> + <maxElementRepetitions>10</maxElementRepetitions> + <maxElementLevels>20</maxElementLevels> + <maxResponseTags>500</maxResponseTags> + <maxResponseSize>500</maxResponseSize> + </settings> + + </capabilities> +</response> Modified: trunk/protocol/examples/inventory_request.xml =================================================================== --- trunk/protocol/examples/inventory_request.xml 2005-12-16 16:31:54 UTC (rev 388) +++ trunk/protocol/examples/inventory_request.xml 2006-01-05 14:51:36 UTC (rev 389) @@ -1,15 +1,15 @@ -<?xml version="1.0" encoding="UTF-8"?> -<request xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" - xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> - <header> - <source accesspoint="http://url_from_client" sendtime="2005-11-11T12:23:56.023+01:00"> - </source> - </header> - <inventory count="false" limit="100" start="0"> - <concepts> - <concept id="http://digir.net/schema/conceptual/darwin/2003/1.0#/Country"/> - <concept id="http://digir.net/schema/conceptual/darwin/2003/1.0#/Genus"/> - </concepts> - </inventory> -</request> +<?xml version="1.0" encoding="UTF-8"?> +<request xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> + <header> + <source accesspoint="http://url_from_client" sendtime="2005-11-11T12:23:56.023+01:00"> + </source> + </header> + <inventory count="false" limit="100" start="0"> + <concepts> + <concept id="http://digir.net/schema/conceptual/darwin/2003/1.0#/Country"/> + <concept id="http://digir.net/schema/conceptual/darwin/2003/1.0#/Genus"/> + </concepts> + </inventory> +</request> Modified: trunk/protocol/examples/inventory_response.xml =================================================================== --- trunk/protocol/examples/inventory_response.xml 2005-12-16 16:31:54 UTC (rev 388) +++ trunk/protocol/examples/inventory_response.xml 2006-01-05 14:51:36 UTC (rev 389) @@ -1,26 +1,26 @@ -<?xml version="1.0" encoding="UTF-8"?> -<response xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" - xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> - <header> - <source accesspoint="http://accesspoint_url" sendtime="2005-11-11T12:23:56.023+01:00"> - <software name="TAPIR_Implementation" version="1.0"/> - </source> - </header> - <inventory> - <concepts> - <concept id="http://digir.net/schema/conceptual/darwin/2003/1.0#/Country"/> - <concept id="http://digir.net/schema/conceptual/darwin/2003/1.0#/Genus"/> - </concepts> - <record> - <value>AUSTRALIA</value> - <value>Calicium</value> - </record> - <record> - <value>AUSTRALIA</value> - <value>Fellhanera</value> - </record> - <summary start="1" totalReturned="2" totalMatched="15" next="3"/> - </inventory> -</response> - +<?xml version="1.0" encoding="UTF-8"?> +<response xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> + <header> + <source accesspoint="http://accesspoint_url" sendtime="2005-11-11T12:23:56.023+01:00"> + <software name="TAPIR_Implementation" version="1.0"/> + </source> + </header> + <inventory> + <concepts> + <concept id="http://digir.net/schema/conceptual/darwin/2003/1.0#/Country"/> + <concept id="http://digir.net/schema/conceptual/darwin/2003/1.0#/Genus"/> + </concepts> + <record> + <value>AUSTRALIA</value> + <value>Calicium</value> + </record> + <record> + <value>AUSTRALIA</value> + <value>Fellhanera</value> + </record> + <summary start="1" totalReturned="2" totalMatched="15" next="3"/> + </inventory> +</response> + Added: trunk/protocol/examples/inventory_template1.xml =================================================================== --- trunk/protocol/examples/inventory_template1.xml 2005-12-16 16:31:54 UTC (rev 388) +++ trunk/protocol/examples/inventory_template1.xml 2006-01-05 14:51:36 UTC (rev 389) @@ -0,0 +1,17 @@ +<?xml version="1.0" encoding="UTF-8"?> +<inventoryTemplate xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> + <label>Specimen Names</label> + <documentation>Search and list unique scientific names of specimen identifications ordered + alphabetically by their name. The filter parameter "name" is used to build the filter</documentation> + <filter> + <like> + <concept id="dwc:/ScientificName"/> + <parameter name="name"/> + </like> + </filter> + <concepts> + <concept id="dwc:/ScientificName"/> + </concepts> +</inventoryTemplate> Modified: trunk/protocol/examples/metadata_request.xml =================================================================== --- trunk/protocol/examples/metadata_request.xml 2005-12-16 16:31:54 UTC (rev 388) +++ trunk/protocol/examples/metadata_request.xml 2006-01-05 14:51:36 UTC (rev 389) @@ -1,13 +1,13 @@ -<?xml version="1.0" encoding="UTF-8"?> - -<request xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" - xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> - <header> - <source accesspoint="http://url_from_client" sendtime="2005-11-11T12:23:56.023+01:00"> - </source> - </header> - <metadata/> -</request> +<?xml version="1.0" encoding="UTF-8"?> + +<request xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> + <header> + <source accesspoint="http://url_from_client" sendtime="2005-11-11T12:23:56.023+01:00"> + </source> + </header> + <metadata/> +</request> Modified: trunk/protocol/examples/metadata_response.xml =================================================================== --- trunk/protocol/examples/metadata_response.xml 2005-12-16 16:31:54 UTC (rev 388) +++ trunk/protocol/examples/metadata_response.xml 2006-01-05 14:51:36 UTC (rev 389) @@ -1,30 +1,30 @@ -<?xml version="1.0" encoding="UTF-8"?> - -<response xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" - xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> - <header> - <source accesspoint="http://accesspoint_url" sendtime="2005-11-11T12:23:56.023+01:00"> - <software name="TAPIR_Implementation" version="1.0"/> - </source> - </header> - <metadata> - <label>Example Data Provider</label> - <accesspoint>http://accesspoint_url</accesspoint> - <relatedEntities> - <entity> - <name>TAPIR working group</name> - <contact> - <name>Frank Sinatra</name> - <email>f.sinatra(a)tapir.com</email> - <role>system administrator</role> - </contact> - <role>Hosting Provider</role> - </entity> - </relatedEntities> - </metadata> -</response> - +<?xml version="1.0" encoding="UTF-8"?> + +<response xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> + <header> + <source accesspoint="http://accesspoint_url" sendtime="2005-11-11T12:23:56.023+01:00"> + <software name="TAPIR_Implementation" version="1.0"/> + </source> + </header> + <metadata> + <label>Example Data Provider</label> + <accesspoint>http://accesspoint_url</accesspoint> + <relatedEntities> + <entity> + <name>TAPIR working group</name> + <contact> + <name>Frank Sinatra</name> + <email>f.sinatra(a)tapir.com</email> + <role>system administrator</role> + </contact> + <role>Hosting Provider</role> + </entity> + </relatedEntities> + </metadata> +</response> + Added: trunk/protocol/examples/output_model1.xml =================================================================== --- trunk/protocol/examples/output_model1.xml 2005-12-16 16:31:54 UTC (rev 388) +++ trunk/protocol/examples/output_model1.xml 2006-01-05 14:51:36 UTC (rev 389) @@ -0,0 +1,66 @@ +<?xml version="1.0" encoding="UTF-8"?> +<outputModel xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" + xmlns:xs="http://www.w3.org/2001/XMLSchema" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation=" + http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd + http://www.w3.org/2001/XMLSchema http://www.w3.org/2001/XMLSchema.xsd + "> + <structure> + <xs:schema targetNamespace="http://example.net/simple_specimen"> + <xs:element name="dataset"> + <xs:complexType> + <xs:sequence> + <xs:element name="specimen" minOccurs="0" maxOccurs="unbounded"> + <xs:complexType> + <xs:sequence> + <xs:element name="identification" maxOccurs="unbounded" + minOccurs="0"> + <xs:complexType> + <xs:sequence> + <xs:element name="name"> + <xs:complexType> + <xs:simpleContent> + <xs:extension base="xs:string"> + <xs:attribute name="family" + type="xs:string"/> + </xs:extension> + </xs:simpleContent> + </xs:complexType> + </xs:element> + <xs:element name="identifier" type="xs:string" + minOccurs="0"/> + </xs:sequence> + <xs:attribute name="date" type="xs:string" + use="optional"/> + </xs:complexType> + </xs:element> + </xs:sequence> + <xs:attribute name="catnum" type="xs:int" use="required"/> + </xs:complexType> + </xs:element> + </xs:sequence> + </xs:complexType> + </xs:element> + </xs:schema> + </structure> + <indexingElement path="/dataset/specimen"/> + <mapping> + <node path="/dataset/specimen@catnum"> + <concept id="dwc:/CatalogNumber"/> + </node> + <node path="/dataset/specimen/identification/name"> + <concept id="dwc:/ScientificName"/> + </node> + <node path="/dataset/specimen/identification/name@family"> + <concept id="dwc:/Family"/> + </node> + <node path="/dataset/specimen/identification/identifier"> + <concept id="abcd:/DataSets/DataSet/Units/Unit/Identifications/Identification/Identifier/IdentifierPersonName/PersonName"/> + </node> + <node path="/dataset/specimen/identification@date"> + <concept id="abcd:/DataSets/DataSet/Units/Unit/Identifications/Identification/IdentificationDate/DateText"/> + </node> + </mapping> +</outputModel> + Modified: trunk/protocol/examples/ping_request.xml =================================================================== --- trunk/protocol/examples/ping_request.xml 2005-12-16 16:31:54 UTC (rev 388) +++ trunk/protocol/examples/ping_request.xml 2006-01-05 14:51:36 UTC (rev 389) @@ -1,17 +1,17 @@ -<?xml version="1.0" encoding="UTF-8"?> - -<request xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" - xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> - <header> - <source accesspoint="http://url_from_client" sendtime="2005-11-11T12:23:56.023+01:00"> - <software name="TAPIR_Implementation" version="1.0"/> - </source> - <destination>http://accesspoint_url</destination> - </header> - <ping></ping> -</request> - +<?xml version="1.0" encoding="UTF-8"?> + +<request xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> + <header> + <source accesspoint="http://url_from_client" sendtime="2005-11-11T12:23:56.023+01:00"> + <software name="TAPIR_Implementation" version="1.0"/> + </source> + <destination>http://accesspoint_url</destination> + </header> + <ping/> +</request> + Modified: trunk/protocol/examples/ping_response.xml =================================================================== --- trunk/protocol/examples/ping_response.xml 2005-12-16 16:31:54 UTC (rev 388) +++ trunk/protocol/examples/ping_response.xml 2006-01-05 14:51:36 UTC (rev 389) @@ -1,13 +1,13 @@ -<?xml version="1.0" encoding="UTF-8"?> - -<response xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" - xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" - xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> - <header> - <source accesspoint="http://accesspoint_url" sendtime="2005-11-11T12:23:57.023+01:00"></source> - </header> - <pong/> -</response> - +<?xml version="1.0" encoding="UTF-8"?> + +<response xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> + <header> + <source accesspoint="http://accesspoint_url" sendtime="2005-11-11T12:23:57.023+01:00"></source> + </header> + <pong/> +</response> + Added: trunk/protocol/examples/search_template1.xml =================================================================== --- trunk/protocol/examples/search_template1.xml 2005-12-16 16:31:54 UTC (rev 388) +++ trunk/protocol/examples/search_template1.xml 2006-01-05 14:51:36 UTC (rev 389) @@ -0,0 +1,19 @@ +<?xml version="1.0" encoding="UTF-8"?> +<searchTemplate xmlns="http://www.tdwg.org/schemas/tapir/datasource/1.0" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://www.tdwg.org/schemas/tapir/datasource/1.0 http://ww2.biocase.org/svn/tapir/trunk/protocol/tapir.xsd"> + <label>Specimen by Name</label> + <documentation>Search specimens by their scientific name. Result is returned ordered by name and + catalogue number. The filter parameter "name" is used to build the filter.</documentation> + <filter> + <like> + <concept id="dwc:/ScientificName"/> + <parameter name="name"/> + </like> + </filter> + <outputModel location="http://the.same.host/output_model1.xml"/> + <orderBy> + <concept id="dwc:/ScientificName"/> + <concept id="dwc:/CatalogNumber"/> + </orderBy> +</searchTemplate>

1 0

[tdwg-tapir] reload / expire external resources
by "Döring, Markus" 05 Jan '06

05 Jan '06

Hello everyone, I am in the process of scanning the schema for errors and update our examples in the SVN. Currently there's a reload attribute in the searchTemplateGroup used by the inline search operation & the template definition. It doesnt really make sense for the template document, does it? How should we handle this in general, maybe for all external resources? a) dont do anything about caching. remove reload/expire attributes b) allow external documents to specify an expiry date (like http) c) add optional reload attribute to all references to external docs In my implementation it was quite important to cache external documents that 1) need to be parsed/processed, like the xml schema docs and 2) that are not accessible all the time. So I think I am in favor of b) and c) actually. The requestor can always specify a reload and the external resource provider can specify a likely expiry date. Does this make sense or just complicates things? Markus

3 2

[tdwg-tapir] output model docs
by "Döring, Markus" 05 Jan '06

05 Jan '06

Hi, for search and inventory templates we allowed to optionally specify a name and a documentation string. This is not possible for output models but I think we should treat them the same way, shouldn't we? Markus

2 1

Re: RDF query and inference in a distributed environment
by Kevin Richards 05 Jan '06

05 Jan '06

Coming from an IT background rather than a taxonomic background, I have never understood the strong "ownership" of data that people/scientists have for their data. This seems rather short sited to me - people with these concerns must have some thought about how to maintain/expose/use "their" data in the long term future? I can understand their concerns, but there must be a solution, otherwise their data will be no-ones concern in 50 years time when it disappears from existence. Another thought I had about data caching systems. Say you want to search the cached/centralised copy of the data (eg a GBIF cache). A list of results is returned, then you decide you want to view more details of one of the results, so you follow a link off to the associated data (this would theoretically be by using the GUID system we are discussing). This would result in viewing the details of the selected record at the location where the GUID resolves to - this would always be the same location as a GUID only resolves to a single location. Is this correct, or would the intention here be to view the cached details of the selected record (which would require an separate ID for all the cached records)? Its this navigation through the caches/repositories that I am not quite sure of how it will work? Kevin >>> deepreef(a)BISHOPMUSEUM.ORG 5/01/2006 8:22 a.m. >>> Wow! Lots of resposes this morning. Many thanks to all who responded! Some replies: Patricia wrote: > I agree with you about the logic in this. However accoding to my daily > experience with potential dataproviders there is a lot of teaching and > convincing needed to make this logic accepted that this does not result > in the loss of control over own data. I agree that to be conviencing > a robust syncronization is needed. I understand the psychology, and agree that the barrier for "expose your data online so that everyone can access it" strikes a more palatable chord than "expose your data online so that everyone can mirror it". > I agree that for machines and storage it is not that expensive. I was > more referring to the human ressources needed to manage the > mirror. Smaller institutions do not have necessary the funds or cannot > justify to their hierachy that staff is devoting time to maint ain a > full miror containing mainly "references" to information coming from > other institutions, but it is easier to justify the time spent to > contribute to the whole with the part concerning directly the institution ... Agreed -- but I wonder really how much more time would be involved in setting up an automated mirror, compared with, e.g., a DiGIR provider. In my earlier post when I said that barriers would likely be technical, I didn't mean that technology doesn't exist (it does). Rather, I meant that in order to be successful, setting up and maintaining a mirror would have to be no more technically challenging than establishing a DiGIR provider. I'm not sure that's possible (yet). > Yes I agree with you that rob ust syncronization will be needed but > as my IT colleague always remind me, I guess we must not forget that > setting up an IT infrastructure is most of the time 10 % technical > issues to be solved and 90% of the time solving "human problems > and barriers" to make it work and accepted ... I agree -- but part of overcoming the human problems and barriers is to make setting up a mirror site relatively easy for a non-specialist IT person. And this is really a technical challenge. The other human problems & barriers (IPR, allowing access to hard-earned data, confidence that data edit authorization rules will be enforced, etc.) also have some technical foundation, and also exist for the distributed/complementary approach. Ricardo wrote: > Instead of mutually exclusive, both approaches you mention are > complementary. The "distributed complementary data" aproach is a > fundamental part of the infrastructure necessary to build the > "distributed mirror copies" you propose. That (the "distributed > complementary data" approach) is essentially what we have in place now > with DiGIR/BioCase/Tapir as the harvesting protocol and GBIF and other > institutions as harvesters. The only missing piece of software we need > to really have "an automated and robust synchronization protocol", in my > opinion is some kind of push mechanism to trigger updates in the caches. Agreed on all points. > However, I think it is not very useful to try to standardize how the > distributed mirror copies should be built and organized. Here's where I disagree somewhat. Standardized mirror protocols would be a fundamental necessity of the paradigm I described. The paradigm would simply not work without such standardization. Non-standard means non-interactive, and therefore no automated synchronization. What makes it useful is that every participating organization has local (=high performance) access to a full biodiversity dataset. Another thing that makes it useful is that, rather than needing *all* data providers to be fully functional at any given moment for 100% data retrival, only one mirror site needs to be functional. I understand this is the role that GBIF already serves -- and perhaps that's all that's really needed. > A (socially) > descentralized schema would work better in this case: data providers > would make available data they create and that is under their direct > custody and individual harvesters would be free to look at the metadata > being served and create their own caches, selectively harvesting only > the information that is relevant to the services they intend to provide. There really should be no difference, as long as data edit authorization protocols are as robust as the synchronization protocols. If I'm exposing my specimen data for the world to access (e.g., via DiGIR/BioCase/Tapir), then it shouldn't bother me that GBIF is mirroring it -- as long as GBIF includes metadata about data ownership, and as long as the only people who can edit the data are people who I authorize to do so. And if GBIF is mirroring it, what difference is there if there are 100, or 1000, or 10,000 other servers mirroring it (same assumptions). The two things that are important to me are: 1) corrections to data get propagated quickly to mirrors; and 2) only people who I authorize to edit my data may do so. Other than that, it doesn't really matter if the data are only on my server's hard drive, or are replicated on GBIF's hard drive, or are replicated on 10,000 server hard drives. But the 10,000 hard drives option gives me two advantages: 1) confidence that the data will survive even continental-scale catastrophies; and 2) it means that my server's hard drive (one of the 10,000) has local and immediate access to everyone else's data (for use as authorities, etc.) Rod Page wrote: [an excellent and thought-proviking post, making important, if uncomfortable, observations] > What is present in bioinformatics are large numbers of "value added" > databases that take GenBank, PubMed, etc. and do neat things with > them. This is possible because you can download the entire database. Exactly! The real value of portals to end users are the value-added "neat things". I think we ought to encourage as many enterprising biology/data nerds as possible to build such things. > Each one of these value added databases does need to deal with the > issue of what happens when GenBank (say) changes, but because GenBank > has well defined releases, essentially they can grab a new copy of the > data, update their local copy, and regenerate their database. Wouldn't it be so much nicer if the protocols were in place to automaticaly propagate updates to the value-added sites in real time, so that the aggregated databases were effectively self-maintaining? I guess the fundamental question of this whole discussion ultimately boils down to "push" vs. "pull" paradigms. > In summary, I think the issue raised by Rich is important, but is one > to be addressed by whoever takes on the task of assembling a data > warehouse from the individual providers. Of course, once providers make > their data available, anybody can do this... Yes, but aybody (everybody) has to rebuild it themselves from scratch. What I'm suggesting is to use the TDWG/GBIF/GUID standard community to come up with standard protocols to lower the technological bar of becomming an aggregator -- so much so that it becomes the real foundation of data distribution, dissemination, and access (and makes it feasible for the enterprising biology/data nerds to make good use of their late night hours....) In a later post, Rod wrote: > I think there is a continuum of possibilities: > > 1. Pure distributed - query always sent to remote sources, no data ever > held locally > > 2. Distributed with cache - results from a user query are cached > (either for a limited time, or until source sends a message saying its > data has been updated). > > 3. Distributed with partial local copy - some information harvested > from sources stored locally (e.g., metadata), detailed information only > held by sources. > > 4. Not distributed (data warehouse) - all data from distributed sources > held locally (harvested), with periodic updates. I would add to this list: 5. Distributed Warehouses -- all data is mirrored across many warehouses, automatically kept in synchronization in real time. Queries are sent to any one warehouse. Also, Rod's reference to "The Cathedral and the Bazaar" is excellent and highly relevant. The example of the Hawaii Biological Survey is a good example of this in our community. Chuck Miller wrote: > To cache or mirror data in multiple locations, the dilemma lies > in the simplest question: "Where is my data?" A cache or mirror > from a technologist's perspective is just a technical trick, identical > to the original, dependent upon the master version, no big deal. > But, simplistically, the data may be actually located in another > country, no matter how it got there, and some politicians could m > isconstrue the whole thing, if not properly negotiated and agreed > upon in advance. In my experience, negotiating such agreements > is more work than the technical development. I agree the political issues may be show-stoppers -- but mostly because of unwarranted paranoia on the part of the data owners (unfortunate, but certainly real). But the reality is, any data exposed on a distributed provider (indeed, any data published in any form -- electronic or paper) is already "out there", and located in many different countries. Data contained in a printed publication simulatnously exist in many countries. The only difference with the distributed warehouse approach is that the data are much, MUCH more powerful. Perhaps the real political barrier is putting so much power into so many hands. Sort of like the World Wide Web itself. > I think the distributed nature of DiGIR was critical to selling > it at the start of GBIF. The original design assured providers that > their source data would "stay" in their country and not be wholesale > copied somewhere else. It's hard to say what the political effect > of creating mirrors would be. Agreed on all points. There was a time when the world wasn't even even ready for the DiGIR approach. Perhaps it's too soon to push it to the next level. Bear in mind, though, that all these IPR/political issues relate mostly to specimen data. The real value of the "distributed warehouses" approach to the warehouses providers themselves are the things that would traditionally be though of as "authorities" (taxon names, reference citations, geographic gazetteers, etc.) Maybe the distributed warehouse approach will work well for these kinds of data, but not for specimen data? O.K., I should probably stop here and get to work.... Aloha, Rich Richard L. Pyle, PhD Database Coordinator for Natural Sciences and Associate Zoologist in Ichthyology Department of Natural Sciences, Bishop Museum 1525 Bernice St., Honolulu, HI 96817 Ph: (808)848-4115, Fax: (808)847-8252 email: deepreef(a)bishopmuseum.org http://hbs.bishopmuseum.org/staff/pylerichard.html ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ WARNING: This email and any attachments may be confidential and/or privileged. They are intended for the addressee only and are not to be read, used, copied or disseminated by anyone receiving them in error. If you are not the intended recipient, please notify the sender by return email and delete this message and any attachments. The views expressed in this email are those of the sender and do not necessarily reflect the official views of Landcare Research. Landcare Research http://www.landcareresearch.co.nz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

1 0