Hi Paul,<div><br></div><div>This looks like a good API, but it is yet another API.</div><div><br></div><div>There are at least four ways people tell the world about their linked data.</div><div><br></div><div>1) Set it up so it can be crawled</div>
<div>2) Publish a Semantic SiteMap.</div><div>3) Expose a void.rdf file</div><div>4) Add it to the CKAN package site.</div><div><br></div><div>I do all of these and that does not count all the various GNI, GBIF, EoL, Digger etc. API's.</div>
<div><br></div><div>I don't mean to say that one of these is better than the others. There are advantages to each - but there are simply too many.</div><div><br></div><div>Respectfully,</div><div><br></div><div>- Pete</div>
<div><br><br><div class="gmail_quote">On Wed, Jan 5, 2011 at 9:05 PM, Paul Murray <span dir="ltr"><<a href="mailto:pmurray@anbg.gov.au">pmurray@anbg.gov.au</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div style="word-wrap:break-word"><div class="im"><br><div><div>On 06/01/2011, at 1:36 PM, Peter DeVries wrote:</div><br><br><blockquote type="cite"><div>Most groups now make their data available as an RDF dump which eliminated the need to crawl if you want to pull in a lot of data.</div>
</blockquote><br></div></div><div>We supply this functionality via an OAI-PMH service. The "Protocol for Metadata Harvesting" is a standard way of asking for a list of ids or a list of actual records, limited by an optional "set" specifier and by start/end modification dates. Essentially: "What records do you have?" and "What's changed since last thursday?"</div>
<div><br></div><div>If you pass set specifier to our service, we treat it as the prefix of the LSIDs that you want to limit your search to, allowing data harvesting to be done in manageable chunks.</div><div><br></div><div>
Our implementation does not perform especially well (I think the XSLT engine is the bottleneck). But that aside, OAI-PMH seems to be a sensible way for data stores to maintain mirrors of each other's data. Even just the list of ids changed since X id useful - you can remove them from your cache, and load them via linked data when required.</div>
<div><br></div><div>See:</div><div><span style="white-space:pre-wrap"> <a href="http://www.openarchives.org/OAI/openarchivesprotocol.html" target="_blank">http://www.openarchives.org/OAI/openarchivesprotocol.html</a></span></div>
<div><span style="white-space:pre-wrap"> </span><a href="http://biodiversity.org.au/oaipmh?verb=Identify" target="_blank">http://biodiversity.org.au/oaipmh?verb=Identify</a></div><div><br></div><div>Please do not send ListIdentifiers or ListRecords requests to this service without specifying a set that includes at least one character of the objectid part of the lsid. </div>
<div><span style="white-space:pre-wrap"> </span><a href="http://biodiversity.org.au/oaipmh?verb=ListIdentifiers&metadataPrefix=rdf&set=urn:lsid:biodiversity.org.au:afd.publication:11" target="_blank">http://biodiversity.org.au/oaipmh?verb=ListIdentifiers&metadataPrefix=rdf&set=urn:lsid:biodiversity.org.au:afd.publication:11</a></div>
<div><br></div><div>It hasn't crashed for weeks now, and it would be nice to keep it that way for just a little longer <sob!>.</div><div class="im"><div><br></div><div><span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><div>
<div><font size="1"><span style="font-size:9px"><span style="font-size:medium">_______________________________________________</span></span></font></div></div></span>
</div>
<br><p>If you have received this transmission in error please notify us immediately by return e-mail and delete all copies. If this e-mail or any attachments have been sent to you in error, that error does not constitute waiver of any confidentiality, privilege or copyright in respect of information in the e-mail or attachments.
Please consider the environment before printing this email.</p>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>---------------------------------------------------------------<br>Pete DeVries<br>Department of Entomology<br>University of Wisconsin - Madison<br>445 Russell Laboratories<br>
1630 Linden Drive<br>Madison, WI 53706<br><a href="http://www.taxonconcept.org/" target="_blank">TaxonConcept Knowledge Base</a> / <a href="http://lod.geospecies.org/" target="_blank">GeoSpecies Knowledge Base</a><br><a href="http://about.geospecies.org/" target="_blank">About the GeoSpecies Knowledge Base</a><br>
------------------------------------------------------------<br>
</div>