Hi Paul,
This looks like a good API, but it is yet another API.
There are at least four ways people tell the world about their linked data.
1) Set it up so it can be crawled 2) Publish a Semantic SiteMap. 3) Expose a void.rdf file 4) Add it to the CKAN package site.
I do all of these and that does not count all the various GNI, GBIF, EoL, Digger etc. API's.
I don't mean to say that one of these is better than the others. There are advantages to each - but there are simply too many.
Respectfully,
- Pete
On Wed, Jan 5, 2011 at 9:05 PM, Paul Murray pmurray@anbg.gov.au wrote:
On 06/01/2011, at 1:36 PM, Peter DeVries wrote:
Most groups now make their data available as an RDF dump which eliminated the need to crawl if you want to pull in a lot of data.
We supply this functionality via an OAI-PMH service. The "Protocol for Metadata Harvesting" is a standard way of asking for a list of ids or a list of actual records, limited by an optional "set" specifier and by start/end modification dates. Essentially: "What records do you have?" and "What's changed since last thursday?"
If you pass set specifier to our service, we treat it as the prefix of the LSIDs that you want to limit your search to, allowing data harvesting to be done in manageable chunks.
Our implementation does not perform especially well (I think the XSLT engine is the bottleneck). But that aside, OAI-PMH seems to be a sensible way for data stores to maintain mirrors of each other's data. Even just the list of ids changed since X id useful - you can remove them from your cache, and load them via linked data when required.
See: http://www.openarchives.org/OAI/openarchivesprotocol.html http://biodiversity.org.au/oaipmh?verb=Identify
Please do not send ListIdentifiers or ListRecords requests to this service without specifying a set that includes at least one character of the objectid part of the lsid.
http://biodiversity.org.au/oaipmh?verb=ListIdentifiers&metadataPrefix=rd...
It hasn't crashed for weeks now, and it would be nice to keep it that way for just a little longer <sob!>.
If you have received this transmission in error please notify us immediately by return e-mail and delete all copies. If this e-mail or any attachments have been sent to you in error, that error does not constitute waiver of any confidentiality, privilege or copyright in respect of information in the e-mail or attachments. Please consider the environment before printing this email.