[tdwg-tapir] Proposal
Dear all,
I would like to propose a set of changes to the protocol trying to address the main issues that we've recently discussed here. The changes are described below and if you want to browse the corresponding schema you can find it here:
http://www.cria.org.br/~renato/tapir.xsd
The main issues are related to TapirLite (how to accomodate it without making generic clients overly complicated), and are also related to Donald's suggestion about inventory templates. All other changes proposed were a mere consequence of browsing the schema and finding parts that I thought could be improved.
Please, take a look when you have some time and let me know if you have any comments, suggestions, objections, etc. Personally I'm happy with the changes, so if you all agree, and if we don't find other issues, I'll suggest to "feature freeze" it as soon as possible so that people can (re)start implementation based on it.
The feature freeze was originally planned for yesterday (December 15th), but since the current version still has open issues and this proposal should obviously be reviewed and discussed, I'll suggest that we postpone it for at least January 5th. By the way, I'll be on vacation during the next two weeks, so only accessing e-mail sporadically.
Best Regards, -- Renato
------- TAPIR / proposed changes -------
* Change filter capabilities so that implementations either accept all filter functionality or none. Specific changes: rename element /response/capabilities/operators to "filter" and include an optional subelement called "encoding". Rename "boolean" to "booleanOperators", and make all logical and comparative operators mandatory. This way parsers can always rely on the existence of a <filter> element and then pass it to a specific handler (same technique suggested by Steve in other parts of the protocol). Such handler can just detect if there's an element "encoding" or not, which means "I understand the basic filtering mechanism" or "I don't know how to parse any filters".
TapirLite implementations will answer in the capabilities: ... <filters/> ...
Full TAPIR implementations will answer: ... <filters> <encoding> <expression> <concept/> <literal/> <parameter/> <variable/> <arithmetic/> </expression> <booleanOperators> <logical> <not/> <and/> <or/> </logical> <comparative> <equals/> <greaterThan/> <greaterThanOrEquals/> <lessThan/> <lessThanOrEquals/> <in/> <isNull/> <like/> </comparative> </booleanOperators> </encoding> </filters> ...
* Change multiCOPType (basis of the <in> operator) so that <values> can contain a sequence of "expression" elements instead of only literals. This will allow parameters with multiple values (or multiple parameters) to be used by the TCS API.
* Re-include an operation completely dedicated to the usage of queryTemples (I called it "view" but I'm happy with any other better names if people feel uncomfortable with it). This operation would only be accessible through GET whithout using XML requests. And it would not accept the "filter" and "partial" parameters. This seems to be the cleanest way to accomodate TapirLite since the _only_ change in the schema is the new optional element /response/capabilities/operations/view indicating the query templates supported. Obs: I tried other ways, but the results were not so nice and the changes were more drastic.
* Remove the possibility to use queryTemplateURI in XML search requests. If someone wants to use a queryTemplateURI in a search request, it's so much simpler to use regular GET parameters in the URL that it's looks reasonable to simplify the protocol in these situations.
* Move /response/capabilities/operations/search/queryTemplates to /response/capabilities/operations/view/queryTemplates.
* On the other hand, include a new <predefinedOutputModels> element in /response/capabilities/operations/search (issue raised in a recent discussion with Rob in this mailing list). DiGIR2 will make use of it.
* Reduce "queryTemplateGroup" to contain only <label> and <documentation>. Move the other elements to a new "searchTemplateGroup" which also references the "queryTemplateGroup". Create a new "inventoryTemplateGroup" in the same way, and make the inventory request element reference it (as it happens in the search request definition). Remove global element <queryTemplate>, including <searchTemplate> and <inventoryTemplate> to make the schema validate external template definitions. This way query templates used in view operations can point to predefined search or inventory operations.
Other changes:
* Make /response/capabilities/concepts/schema a mandatory element so that any implementation shows which concepts are present.
* Remove controlled vocabulary of element /response/metadata/relatedEntities/entity/contact/role and make it unbounded: controlled vocabularies do help with standardization, but they also reduce flexibility. In different contexts people could easily envision different uses for that attribute, so I would rather enforce vocabulary control in other levels (such as software configuration or network configuration).
* Remove "datum" attribute from geoLocationType: the corresponding element has been included so that "related entities" (part of metadata responses) could indicate their location. People can easily always put lat/long in WGS84 in this case, it's just a matter of adding a note about that in the documentation.
* Rename primaryConceptNameServer and secondaryConceptNameServer to primary and secondary, respectively (the enclosing element already provides the context). Make <primary> optional.
* Rename global element <environment> to <variable>, and move /response/capabilities/search/variables to /response/capabilities/variables since they can be used in both mappings and filters. Remove redefinition of all variables in the capabilities response by referencing the existing variable element defintion. Remove empty /response/capabilities/search/variables/environment/metadata element.
-------
Renato,
Thanks for all this hard work. I'm afraid I now need a little help. I'm no longer sure exactly how a queryTemplate is defined and how the GET request to use one works. Is it possible to provide a small example to help me put it all together?
First (in the capabilities response) what exactly goes into the externalResourceType? Is it intended to contain anything more than the location attribute (i.e. is there some text label in the queryTemplate element body)?
Secondly, what is stored at the URL indicated by the location attribute? I assume that it is some subtree from an inventory or search request but what is the expected form (expected root element)? Can you provide a parameterised example?
Thirdly, what is the expected form for a GET request using a queryTemplate? How is the template specified in the request?
Sorry to be stupid, but this would be a real help in responding accurately.
Happy New Year and very best wishes,
Donald
--------------------------------------------------------------- Donald Hobern (dhobern@gbif.org) Programme Officer for Data Access and Database Interoperability Global Biodiversity Information Facility Secretariat Universitetsparken 15, DK-2100 Copenhagen, Denmark Tel: +45-35321483 Mobile: +45-28751483 Fax: +45-35321480 ---------------------------------------------------------------
-----Original Message----- From: tdwg-tapir-bounces@lists.tdwg.org [mailto:tdwg-tapir-bounces@lists.tdwg.org] On Behalf Of Renato De Giovanni Sent: 16 December 2005 19:18 To: tdwg-tapir@lists.tdwg.org Subject: [tdwg-tapir] Proposal
Dear all,
I would like to propose a set of changes to the protocol trying to address the main issues that we've recently discussed here. The changes are described below and if you want to browse the corresponding schema you can find it here:
http://www.cria.org.br/~renato/tapir.xsd
The main issues are related to TapirLite (how to accomodate it without making generic clients overly complicated), and are also related to Donald's suggestion about inventory templates. All other changes proposed were a mere consequence of browsing the schema and finding parts that I thought could be improved.
Please, take a look when you have some time and let me know if you have any comments, suggestions, objections, etc. Personally I'm happy with the changes, so if you all agree, and if we don't find other issues, I'll suggest to "feature freeze" it as soon as possible so that people can (re)start implementation based on it.
The feature freeze was originally planned for yesterday (December 15th), but since the current version still has open issues and this proposal should obviously be reviewed and discussed, I'll suggest that we postpone it for at least January 5th. By the way, I'll be on vacation during the next two weeks, so only accessing e-mail sporadically.
Best Regards, -- Renato
------- TAPIR / proposed changes -------
* Change filter capabilities so that implementations either accept all filter functionality or none. Specific changes: rename element /response/capabilities/operators to "filter" and include an optional subelement called "encoding". Rename "boolean" to "booleanOperators", and make all logical and comparative operators mandatory. This way parsers can always rely on the existence of a <filter> element and then pass it to a specific handler (same technique suggested by Steve in other parts of the protocol). Such handler can just detect if there's an element "encoding" or not, which means "I understand the basic filtering mechanism" or "I don't know how to parse any filters".
TapirLite implementations will answer in the capabilities: ... <filters/> ...
Full TAPIR implementations will answer: ... <filters> <encoding> <expression> <concept/> <literal/> <parameter/> <variable/> <arithmetic/> </expression> <booleanOperators> <logical> <not/> <and/> <or/> </logical> <comparative> <equals/> <greaterThan/> <greaterThanOrEquals/> <lessThan/> <lessThanOrEquals/> <in/> <isNull/> <like/> </comparative> </booleanOperators> </encoding> </filters> ...
* Change multiCOPType (basis of the <in> operator) so that <values> can contain a sequence of "expression" elements instead of only literals. This will allow parameters with multiple values (or multiple parameters) to be used by the TCS API.
* Re-include an operation completely dedicated to the usage of queryTemples (I called it "view" but I'm happy with any other better names if people feel uncomfortable with it). This operation would only be accessible through GET whithout using XML requests. And it would not accept the "filter" and "partial" parameters. This seems to be the cleanest way to accomodate TapirLite since the _only_ change in the schema is the new optional element /response/capabilities/operations/view indicating the query templates supported. Obs: I tried other ways, but the results were not so nice and the changes were more drastic.
* Remove the possibility to use queryTemplateURI in XML search requests. If someone wants to use a queryTemplateURI in a search request, it's so much simpler to use regular GET parameters in the URL that it's looks reasonable to simplify the protocol in these situations.
* Move /response/capabilities/operations/search/queryTemplates to /response/capabilities/operations/view/queryTemplates.
* On the other hand, include a new <predefinedOutputModels> element in /response/capabilities/operations/search (issue raised in a recent discussion with Rob in this mailing list). DiGIR2 will make use of it.
* Reduce "queryTemplateGroup" to contain only <label> and <documentation>. Move the other elements to a new "searchTemplateGroup" which also references the "queryTemplateGroup". Create a new "inventoryTemplateGroup" in the same way, and make the inventory request element reference it (as it happens in the search request definition). Remove global element <queryTemplate>, including <searchTemplate> and <inventoryTemplate> to make the schema validate external template definitions. This way query templates used in view operations can point to predefined search or inventory operations.
Other changes:
* Make /response/capabilities/concepts/schema a mandatory element so that any implementation shows which concepts are present.
* Remove controlled vocabulary of element /response/metadata/relatedEntities/entity/contact/role and make it unbounded: controlled vocabularies do help with standardization, but they also reduce flexibility. In different contexts people could easily envision different uses for that attribute, so I would rather enforce vocabulary control in other levels (such as software configuration or network configuration).
* Remove "datum" attribute from geoLocationType: the corresponding element has been included so that "related entities" (part of metadata responses) could indicate their location. People can easily always put lat/long in WGS84 in this case, it's just a matter of adding a note about that in the documentation.
* Rename primaryConceptNameServer and secondaryConceptNameServer to primary and secondary, respectively (the enclosing element already provides the context). Make <primary> optional.
* Rename global element <environment> to <variable>, and move /response/capabilities/search/variables to /response/capabilities/variables since they can be used in both mappings and filters. Remove redefinition of all variables in the capabilities response by referencing the existing variable element defintion. Remove empty /response/capabilities/search/variables/environment/metadata element.
-------
_______________________________________________ tdwg-tapir mailing list tdwg-tapir@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-tapir_lists.tdwg.org
Hi Donald,
Thanks for all this hard work. I'm afraid I now need a little help. I'm no longer sure exactly how a queryTemplate is defined and how the GET request to use one works. Is it possible to provide a small example to help me put it all together?
Sure it is possible, I'll try to provide some examples below (sorry I didn't have time to do that before).
First (in the capabilities response) what exactly goes into the externalResourceType? Is it intended to contain anything more than the location attribute (i.e. is there some text label in the queryTemplate element body)?
No, it only contains the location attribute. It's just a pointer to an external definition. Just as it was in the previous version: ... <queryTemplates> <queryTemplate location="http://someUrl/mySearchTemplate.xml" /> <queryTemplate location="http://someUrl/myInventoryTemplate.xml" /> </queryTemplates> ...
Secondly, what is stored at the URL indicated by the location attribute? I assume that it is some subtree from an inventory or search request but what is the expected form (expected root element)? Can you provide a parameterised example?
The URL should point to a search or an inventory template, such as:
<?xml version="1.0"?> <searchTemplate> <label>GetTNamesByID</label> <documentation>Retrieves one or more TaxonNames by their ID field</documentation> <outputModelURI>http://someUrl/model.xml</outputModelURI> <filter> <in> <concept id="TCSnamespace#Dataset/TaxonNames/TaxonName@id" /> <values> <parameter name="id" /> </values> </in> </filter> <orderBy> <concept id="TCSnamespace#Dataset/TaxonNames/TaxonName/Simple" /> </orderBy> </searchTemplate>
note: outputModelURI points to an external definition of an output model (no changes here - it has the same structure defined by the current schema).
or
<?xml version="1.0"?> <inventoryTemplate> <label>GetNamesInventory</label> <documentation>Retrieves a list of distinct Taxon Names </documentation> <concepts> <concept id="TCSnamespace#Dataset/TaxonNames/TaxonName/Simple" /> </concepts> </inventoryTemplate>
note: no filter was used here (it is optional).
Thirdly, what is the expected form for a GET request using a queryTemplate? How is the template specified in the request?
The examples above could be invoked, respectively, with:
http://myTapirProvider/someScript?op=view&template=http://someUrl/mySe archTemplate.xml&id=someLSID&id=someDOI&id=somePURL
http://myTapirProvider/someScript?op=view&template=http://someUrl/myIn ventoryTemplate.xml
Sorry to be stupid, but this would be a real help in responding accurately. Happy New Year and very best wishes,
No worries, it's definitely easier when we can look at concrete examples - I should have prepared them before.
Best Regards and a Happy New Year, -- Renato
Renato,
Many thanks. That's a great help.
Donald
--------------------------------------------------------------- Donald Hobern (dhobern@gbif.org) Programme Officer for Data Access and Database Interoperability Global Biodiversity Information Facility Secretariat Universitetsparken 15, DK-2100 Copenhagen, Denmark Tel: +45-35321483 Mobile: +45-28751483 Fax: +45-35321480 ---------------------------------------------------------------
-----Original Message----- From: tdwg-tapir-bounces@lists.tdwg.org [mailto:tdwg-tapir-bounces@lists.tdwg.org] On Behalf Of Renato De Giovanni Sent: 03 January 2006 22:00 To: tdwg-tapir@lists.tdwg.org Subject: Re: [tdwg-tapir] Proposal
Hi Donald,
Thanks for all this hard work. I'm afraid I now need a little help. I'm
no
longer sure exactly how a queryTemplate is defined and how the GET request to use one works. Is it possible to provide a small example to help me
put
it all together?
Sure it is possible, I'll try to provide some examples below (sorry I didn't have time to do that before).
First (in the capabilities response) what exactly goes into the externalResourceType? Is it intended to contain anything more than the location attribute (i.e. is there some text label in the queryTemplate element body)?
No, it only contains the location attribute. It's just a pointer to an external definition. Just as it was in the previous version: ... <queryTemplates> <queryTemplate location="http://someUrl/mySearchTemplate.xml" /> <queryTemplate location="http://someUrl/myInventoryTemplate.xml" /> </queryTemplates> ...
Secondly, what is stored at the URL indicated by the location attribute?
I
assume that it is some subtree from an inventory or search request but
what
is the expected form (expected root element)? Can you provide a parameterised example?
The URL should point to a search or an inventory template, such as:
<?xml version="1.0"?> <searchTemplate> <label>GetTNamesByID</label> <documentation>Retrieves one or more TaxonNames by their ID field</documentation> <outputModelURI>http://someUrl/model.xml</outputModelURI> <filter> <in> <concept id="TCSnamespace#Dataset/TaxonNames/TaxonName@id" /> <values> <parameter name="id" /> </values> </in> </filter> <orderBy> <concept id="TCSnamespace#Dataset/TaxonNames/TaxonName/Simple" /> </orderBy> </searchTemplate>
note: outputModelURI points to an external definition of an output model (no changes here - it has the same structure defined by the current schema).
or
<?xml version="1.0"?> <inventoryTemplate> <label>GetNamesInventory</label> <documentation>Retrieves a list of distinct Taxon Names </documentation> <concepts> <concept id="TCSnamespace#Dataset/TaxonNames/TaxonName/Simple" /> </concepts> </inventoryTemplate>
note: no filter was used here (it is optional).
Thirdly, what is the expected form for a GET request using a
queryTemplate?
How is the template specified in the request?
The examples above could be invoked, respectively, with:
http://myTapirProvider/someScript?op=view&template=http://someUrl/mySe archTemplate.xml&id=someLSID&id=someDOI&id=somePURL
http://myTapirProvider/someScript?op=view&template=http://someUrl/myIn ventoryTemplate.xml
Sorry to be stupid, but this would be a real help in responding
accurately.
Happy New Year and very best wishes,
No worries, it's definitely easier when we can look at concrete examples - I should have prepared them before.
Best Regards and a Happy New Year, -- Renato
_______________________________________________ tdwg-tapir mailing list tdwg-tapir@lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-tapir_lists.tdwg.org
Hi everybody,
If there are no serious objections or other alternatives to that package of changes that was recently proposed, I'll take the liberty to commit them in the svn repository. If anyone finds any issues, or has any other suggestions, or disagrees with anything, please don't be shy!
I'll try to commit the changes individually just to keep a clean change log. However this will produce several notification messages (sorry!).
Regards, -- Renato
On 16 Dec 2005 at 16:18, Renato De Giovanni wrote:
Dear all,
I would like to propose a set of changes to the protocol trying to address the main issues that we've recently discussed here. The changes are described below and if you want to browse the corresponding schema you can find it here:
http://www.cria.org.br/~renato/tapir.xsd
The main issues are related to TapirLite (how to accomodate it without making generic clients overly complicated), and are also related to Donald's suggestion about inventory templates. All other changes proposed were a mere consequence of browsing the schema and finding parts that I thought could be improved.
Please, take a look when you have some time and let me know if you have any comments, suggestions, objections, etc. Personally I'm happy with the changes, so if you all agree, and if we don't find other issues, I'll suggest to "feature freeze" it as soon as possible so that people can (re)start implementation based on it.
The feature freeze was originally planned for yesterday (December 15th), but since the current version still has open issues and this proposal should obviously be reviewed and discussed, I'll suggest that we postpone it for at least January 5th. By the way, I'll be on vacation during the next two weeks, so only accessing e-mail sporadically.
Best Regards,
Renato
------- TAPIR / proposed changes -------
- Change filter capabilities so that implementations either accept
all filter functionality or none. Specific changes: rename element /response/capabilities/operators to "filter" and include an optional subelement called "encoding". Rename "boolean" to "booleanOperators", and make all logical and comparative operators mandatory. This way parsers can always rely on the existence of a <filter> element and then pass it to a specific handler (same technique suggested by Steve in other parts of the protocol). Such handler can just detect if there's an element "encoding" or not, which means "I understand the basic filtering mechanism" or "I don't know how to parse any filters".
TapirLite implementations will answer in the capabilities: ...
<filters/> ...
Full TAPIR implementations will answer: ...
<filters> <encoding> <expression> <concept/> <literal/> <parameter/> <variable/> <arithmetic/> </expression> <booleanOperators> <logical> <not/> <and/> <or/> </logical> <comparative> <equals/> <greaterThan/> <greaterThanOrEquals/> <lessThan/> <lessThanOrEquals/> <in/> <isNull/> <like/> </comparative> </booleanOperators> </encoding> </filters> ...
- Change multiCOPType (basis of the <in> operator) so that <values>
can contain a sequence of "expression" elements instead of only literals. This will allow parameters with multiple values (or multiple parameters) to be used by the TCS API.
- Re-include an operation completely dedicated to the usage of
queryTemples (I called it "view" but I'm happy with any other better names if people feel uncomfortable with it). This operation would only be accessible through GET whithout using XML requests. And it would not accept the "filter" and "partial" parameters. This seems to be the cleanest way to accomodate TapirLite since the _only_ change in the schema is the new optional element /response/capabilities/operations/view indicating the query templates supported. Obs: I tried other ways, but the results were not so nice and the changes were more drastic.
- Remove the possibility to use queryTemplateURI in XML search
requests. If someone wants to use a queryTemplateURI in a search request, it's so much simpler to use regular GET parameters in the URL that it's looks reasonable to simplify the protocol in these situations.
- Move /response/capabilities/operations/search/queryTemplates to
/response/capabilities/operations/view/queryTemplates.
- On the other hand, include a new <predefinedOutputModels> element
in /response/capabilities/operations/search (issue raised in a recent discussion with Rob in this mailing list). DiGIR2 will make use of it.
- Reduce "queryTemplateGroup" to contain only <label> and
<documentation>. Move the other elements to a new "searchTemplateGroup" which also references the "queryTemplateGroup". Create a new "inventoryTemplateGroup" in the same way, and make the inventory request element reference it (as it happens in the search request definition). Remove global element <queryTemplate>, including <searchTemplate> and <inventoryTemplate> to make the schema validate external template definitions. This way query templates used in view operations can point to predefined search or inventory operations.
Other changes:
- Make /response/capabilities/concepts/schema a mandatory element so
that any implementation shows which concepts are present.
- Remove controlled vocabulary of element
/response/metadata/relatedEntities/entity/contact/role and make it unbounded: controlled vocabularies do help with standardization, but they also reduce flexibility. In different contexts people could easily envision different uses for that attribute, so I would rather enforce vocabulary control in other levels (such as software configuration or network configuration).
- Remove "datum" attribute from geoLocationType: the corresponding
element has been included so that "related entities" (part of metadata responses) could indicate their location. People can easily always put lat/long in WGS84 in this case, it's just a matter of adding a note about that in the documentation.
- Rename primaryConceptNameServer and secondaryConceptNameServer to
primary and secondary, respectively (the enclosing element already provides the context). Make <primary> optional.
- Rename global element <environment> to <variable>, and move
/response/capabilities/search/variables to /response/capabilities/variables since they can be used in both mappings and filters. Remove redefinition of all variables in the capabilities response by referencing the existing variable element defintion. Remove empty /response/capabilities/search/variables/environment/metadata element.
participants (2)
-
Donald Hobern -
Renato De Giovanni