January 2006 - tdwg-tag - lists.tdwg.org

Re: RDF query and inference in a distributed environment
by Richard Pyle 03 Jan '06

03 Jan '06

> I'm a great believer in caching, but it comes equipped with one trivial > technical cost which is not always trivial socially. That is, the data > originator must be willing to enter into a contract (in the software, > not legal, sense) to update the data only at a specific time, that is to > provide a "good until" promise. Not if automated and robust synchronization protocols are established. This is what I meant by "issues of replication/synchronization". Data records would not need to have a pre-determined lifespan. Rather, the lifespan would be defined as "good until updated by an authorized editor". Once an authorized editor updates *any* copy of a given GUID-tagged record, the update would be automatically propagated to all protocol-compliant replicates of that record. Unlike dynamic DNS, most of our records (taxon names, specimen objects, defined & georeferenced localities, publications, etc.) are mostly static and perpetual, changing only when errors or omissions are corrected. Thus, most of the data traffic among replicate copies would be from new record additions, rather than existing record modifications. > Otherwise, a client of the cached > version is clueless whether the data in the cache is accurate. Not if robust automated replication/synchronization protocols are in place. Aloha, Rich

1 0

Re: RDF query and inference in a distributed environment
by Bob Morris 03 Jan '06

03 Jan '06

I'm a great believer in caching, but it comes equipped with one trivial technical cost which is not always trivial socially. That is, the data originator must be willing to enter into a contract (in the software, not legal, sense) to update the data only at a specific time, that is to provide a "good until" promise. Otherwise, a client of the cached version is clueless whether the data in the cache is accurate. Only with some notion of expiration can the cache host determine if a query must induce a cache refresh.Often when I've raised this point with biodiversity data providers, they hold out the position that they have to be allowed to update the data whenever they need to (as opposed, say to waiting a day, a week, or...). I always find this a strange position from a community which tolerates delays of years before , e.g. a taxonomic name is official because that's how long the journal took from submission to publication.... There are several functionally equivalent solutions to the technical problem and this list needn't discuss them (not the least because more than a few readers are probably tired of my carping about this point) Richard Pyle wrote: >>Long term what I think might happen is that users have their own triple >>stores, and as they do queries the results get added to their own >>triple store and they can make inferences locally that they are >>interested in. MIT's Piggy bank project >>(http://simile.mit.edu/piggy-bank/) is an example of this sort of >>approach. >> >> > >With hard drive sizes spiraling skyward, and $/GB ($/TB) spiraling >downward.... I'm wondering whether or not the "distributed" system that >serves us best might be "distributeded mirror copies", rather than >distributed complementary data. I've been pushing this approach for >taxonomic data for a while, but perhaps it would be useful for other shared >data as well (geographic localities, people/agents, publications/references, >etc.) Even for specimen data -- where "ownership" is unambiguous -- it >seems that as long as the ownership is clearly embedded in the core >metadata, there are more fundamental advantages in storing and serving data >from multiple data resources, rather than serving it from only one single >data resource. > >One way to look at it would be "robust caching", with automated update >capabilities. The main benefits would be: > >1) Large-scale distributed backup of the world's biodata (ensuring >perpetuity across a changing technological landscape); >2) Performance and reliability enhancement for local data authority needs; >4) Essentially 100% data availability (like DNS), regardless of which >servers are up or down at any given moment; > > [This is not entirely true of the DNS. If no authoritative server is accessible for a record at the time a DNS record expires, then applications have to just decide whether they will attempt, and accept as trustworthy, whatever answers up on the expired IP address, if indeed there is one available. Not that the DNS attempts to provide any trust model anyway...] >3) Maximization of distributed work/effort for data "maintenance and >repair". > >The point is, the technology discussions would focus less on issues of >distributed queries, and more on issues of replication/synchronization and >data edit authorization protocols. > >Perhaps this would be reaching too far, too soon. But on the other hand, I >don't see why implementing a "distributed mirror" system would be any more >technically, financially, or socially challenging than implementing a >distributed query system for distributed data. > >Aloha, >Rich > >Richard L. Pyle, PhD >Database Coordinator for Natural Sciences > and Associate Zoologist in Ichthyology >Department of Natural Sciences, Bishop Museum >1525 Bernice St., Honolulu, HI 96817 >Ph: (808)848-4115, Fax: (808)847-8252 >email: deepreef(a)bishopmuseum.org >http://hbs.bishopmuseum.org/staff/pylerichard.html > > --Bob Morris -- Robert A. Morris Professor of Computer Science UMASS-Boston ram(a)cs.umb.edu http://www.cs.umb.edu/efg http://www.cs.umb.edu/~ram phone (+1)617 287 6466

1 0

Re: RDF query and inference in a distributed environment
by Roderic Page 03 Jan '06

03 Jan '06

Dear Ricardo, The issues you raise are the ones we are currently playing with at Glasgow. It's still early days, but a here are a few thoughts. In terms of distributed queries, languages such as SPARQL support querying remote sources (for an introduction see Leigh Dodd's article http://www.xml.com/pub/a/2005/11/16/introducing-sparql-querying- semantic-web-tutorial.html). SPARQL has its own query protocol using web services. I've not played with this much yet, but it look promising. Basically, your query can contain the URL of a resource and SPARQL will query it. How this will scale across multiple sources I'm not sure. Another tool to consider is TAP (http://tap.stanford.edu/). In terms of discovery, this is a separate issue in some ways. I think an obvious thing to look at is BioMoby. A light weight version of this would be to have a RDF source that listed data sources and described what kinds of information they contain. A bit like GBIFs UDDI registry, but actually useful ;-) Long term what I think might happen is that users have their own triple stores, and as they do queries the results get added to their own triple store and they can make inferences locally that they are interested in. MIT's Piggy bank project (http://simile.mit.edu/piggy-bank/) is an example of this sort of approach. Regards Rod On 3 Jan 2006, at 13:24, Ricardo Scachetti Pereira wrote: > Dear Rod, > > Thanks for putting those ideas together and sharing them with the > group. We really needed some more concrete examples to explore the use > of LSID and RDF in the GUID metadata. It definitely sound like a good > combination of technologies to explore. > > Reading your manuscript one question kept coming to my mind. The > most important benefits of using RDF seem to be the graph > representation, the possibility of merging separate graphs into larger > ones (the merge operation), and the possibility of making inferences on > the resulting graph. All those features are supported by the so called > triple stores. > > However, the triple store examples I come across always use a local > and centralized architecture. I'm aware that those are simplifications > needed to get the new concepts through, but I suppose we need to > address > the scalability issues of those technologies if we want to use them in > production environment. > > So I wanted to explore a little in this list is the following issue: > How does the RDF inference mechanisms scale up to a distributed > environment like ours? In my opinion, GUID technologies provide only a > bit (although a fundamental bit) of the infrastructure needed to make > the RDF metadata framework fully functional, i.e., it provides the > function: get me all metadata for this ID (besides persistence and > maybe > provenance). I suppose that other pieces of functionality will be > required, such as: i) other metadata discovery mechanisms (to find out, > for example, who has metadata using this or that ontology); ii) a more > flexible search mechanism (such as RDQL or SPARQL). In other words, we > might need a RDF compatible query protocol. > > Then it makes a lot of sense to have "aggregators" all around (such > as GBIF or others) harvesting and merging RDF triples of interest into > local triple stores and providing query and inference services, and > providing links (for humans and machines) back to the information > source. > > Does anyone have an idea on how SPARQL or related technologies are > right now in terms of supporting this kind of functionality? > > Cheers, > > Ricardo > > > Roderic Page wrote: > >> I've finally managed to put down some ideas on LSIDs, metadata, and >> taxonomic names in the form of a manuscript. It's a bit rough, but I >> thought members of this list might find it of interest. You can grab a >> PDF here: >> http://darwin.zoology.gla.ac.uk/~rpage/lsid/examples/lsid.pdf >> >> The web site http://darwin.zoology.gla.ac.uk/~rpage/lsid/examples has >> links to some example RDF files that are mentioned in the manuscript >> (where they are trimmed to save space). >> >> I'd welcome any comments. >> >> Regards >> >> Rod >> >> >> >> ---------------------------------------------------------------------- >> -- >> ---------------------------------------- >> Professor Roderic D. M. Page >> Editor, Systematic Biology >> DEEB, IBLS >> Graham Kerr Building >> University of Glasgow >> Glasgow G12 8QP >> United Kingdom >> >> Phone: +44 141 330 4778 >> Fax: +44 141 330 2792 >> email: r.page(a)bio.gla.ac.uk >> web: http://taxonomy.zoology.gla.ac.uk/rod/rod.html >> reprints: http://taxonomy.zoology.gla.ac.uk/rod/pubs.html >> >> Subscribe to Systematic Biology through the Society of Systematic >> Biologists Website: http://systematicbiology.org >> Search for taxon names at >> http://darwin.zoology.gla.ac.uk/~rpage/portal/ >> Find out what we know about a species at http://ispecies.org >> > > <ricardo.vcf> ------------------------------------------------------------------------ ---------------------------------------- Professor Roderic D. M. Page Editor, Systematic Biology DEEB, IBLS Graham Kerr Building University of Glasgow Glasgow G12 8QP United Kingdom Phone: +44 141 330 4778 Fax: +44 141 330 2792 email: r.page(a)bio.gla.ac.uk web: http://taxonomy.zoology.gla.ac.uk/rod/rod.html reprints: http://taxonomy.zoology.gla.ac.uk/rod/pubs.html Subscribe to Systematic Biology through the Society of Systematic Biologists Website: http://systematicbiology.org Search for taxon names at http://darwin.zoology.gla.ac.uk/~rpage/portal/ Find out what we know about a species at http://ispecies.org

1 0

RDF query and inference in a distributed environment
by Ricardo Scachetti Pereira 03 Jan '06

03 Jan '06

Dear Rod, Thanks for putting those ideas together and sharing them with the group. We really needed some more concrete examples to explore the use of LSID and RDF in the GUID metadata. It definitely sound like a good combination of technologies to explore. Reading your manuscript one question kept coming to my mind. The most important benefits of using RDF seem to be the graph representation, the possibility of merging separate graphs into larger ones (the merge operation), and the possibility of making inferences on the resulting graph. All those features are supported by the so called triple stores. However, the triple store examples I come across always use a local and centralized architecture. I'm aware that those are simplifications needed to get the new concepts through, but I suppose we need to address the scalability issues of those technologies if we want to use them in production environment. So I wanted to explore a little in this list is the following issue: How does the RDF inference mechanisms scale up to a distributed environment like ours? In my opinion, GUID technologies provide only a bit (although a fundamental bit) of the infrastructure needed to make the RDF metadata framework fully functional, i.e., it provides the function: get me all metadata for this ID (besides persistence and maybe provenance). I suppose that other pieces of functionality will be required, such as: i) other metadata discovery mechanisms (to find out, for example, who has metadata using this or that ontology); ii) a more flexible search mechanism (such as RDQL or SPARQL). In other words, we might need a RDF compatible query protocol. Then it makes a lot of sense to have "aggregators" all around (such as GBIF or others) harvesting and merging RDF triples of interest into local triple stores and providing query and inference services, and providing links (for humans and machines) back to the information source. Does anyone have an idea on how SPARQL or related technologies are right now in terms of supporting this kind of functionality? Cheers, Ricardo Roderic Page wrote: > I've finally managed to put down some ideas on LSIDs, metadata, and > taxonomic names in the form of a manuscript. It's a bit rough, but I > thought members of this list might find it of interest. You can grab a > PDF here: http://darwin.zoology.gla.ac.uk/~rpage/lsid/examples/lsid.pdf > > The web site http://darwin.zoology.gla.ac.uk/~rpage/lsid/examples has > links to some example RDF files that are mentioned in the manuscript > (where they are trimmed to save space). > > I'd welcome any comments. > > Regards > > Rod > > > > ------------------------------------------------------------------------ > ---------------------------------------- > Professor Roderic D. M. Page > Editor, Systematic Biology > DEEB, IBLS > Graham Kerr Building > University of Glasgow > Glasgow G12 8QP > United Kingdom > > Phone: +44 141 330 4778 > Fax: +44 141 330 2792 > email: r.page(a)bio.gla.ac.uk > web: http://taxonomy.zoology.gla.ac.uk/rod/rod.html > reprints: http://taxonomy.zoology.gla.ac.uk/rod/pubs.html > > Subscribe to Systematic Biology through the Society of Systematic > Biologists Website: http://systematicbiology.org > Search for taxon names at http://darwin.zoology.gla.ac.uk/~rpage/portal/ > Find out what we know about a species at http://ispecies.org >

1 0

Re: LSID manuscript
by Roger Hyam 03 Jan '06

03 Jan '06

Thanks Rod. It makes a good case for LSID + RDF. Having spent a big chunk of last year working on the Taxon Concept Schema I am itching to expand on some of your points with regard to TaxonConcepts/TaxonNames. There are some issues there that I think we have already covered. But this isn't the place to get into the niceties of taxonomic nomenclature. We have worked pretty hard on reaching what we believe is a consensus ontology on nomenclature and taxon concepts. Although it is expressed in XML Schema at the moment I think it would quite simply transfer to an RDF or OWL Lite ontology and I have started tentative work on this. Perhaps it will be something you can contribute to. Under 3.4 you could mention vCard as I have seen that expressed in RDF for data about people. Happy New Year, Roger Roderic Page wrote: > I've finally managed to put down some ideas on LSIDs, metadata, and > taxonomic names in the form of a manuscript. It's a bit rough, but I > thought members of this list might find it of interest. You can grab a > PDF here: http://darwin.zoology.gla.ac.uk/~rpage/lsid/examples/lsid.pdf > > The web site http://darwin.zoology.gla.ac.uk/~rpage/lsid/examples has > links to some example RDF files that are mentioned in the manuscript > (where they are trimmed to save space). > > I'd welcome any comments. > > Regards > > Rod > > > > ------------------------------------------------------------------------ > ---------------------------------------- > Professor Roderic D. M. Page > Editor, Systematic Biology > DEEB, IBLS > Graham Kerr Building > University of Glasgow > Glasgow G12 8QP > United Kingdom > > Phone: +44 141 330 4778 > Fax: +44 141 330 2792 > email: r.page(a)bio.gla.ac.uk > web: http://taxonomy.zoology.gla.ac.uk/rod/rod.html > reprints: http://taxonomy.zoology.gla.ac.uk/rod/pubs.html > > Subscribe to Systematic Biology through the Society of Systematic > Biologists Website: http://systematicbiology.org > Search for taxon names at http://darwin.zoology.gla.ac.uk/~rpage/portal/ > Find out what we know about a species at http://ispecies.org > -- ------------------------------------- Roger Hyam Technical Architect Taxonomic Databases Working Group ------------------------------------- http://www.tdwg.org roger(a)tdwg.org +44 1578 722782 -------------------------------------

1 0

Re: RDF query and inference in a distributed environment
by Richard Pyle 03 Jan '06

03 Jan '06

> Long term what I think might happen is that users have their own triple > stores, and as they do queries the results get added to their own > triple store and they can make inferences locally that they are > interested in. MIT's Piggy bank project > (http://simile.mit.edu/piggy-bank/) is an example of this sort of > approach. With hard drive sizes spiraling skyward, and $/GB ($/TB) spiraling downward.... I'm wondering whether or not the "distributed" system that serves us best might be "distributeded mirror copies", rather than distributed complementary data. I've been pushing this approach for taxonomic data for a while, but perhaps it would be useful for other shared data as well (geographic localities, people/agents, publications/references, etc.) Even for specimen data -- where "ownership" is unambiguous -- it seems that as long as the ownership is clearly embedded in the core metadata, there are more fundamental advantages in storing and serving data from multiple data resources, rather than serving it from only one single data resource. One way to look at it would be "robust caching", with automated update capabilities. The main benefits would be: 1) Large-scale distributed backup of the world's biodata (ensuring perpetuity across a changing technological landscape); 2) Performance and reliability enhancement for local data authority needs; 4) Essentially 100% data availability (like DNS), regardless of which servers are up or down at any given moment; 3) Maximization of distributed work/effort for data "maintenance and repair". The point is, the technology discussions would focus less on issues of distributed queries, and more on issues of replication/synchronization and data edit authorization protocols. Perhaps this would be reaching too far, too soon. But on the other hand, I don't see why implementing a "distributed mirror" system would be any more technically, financially, or socially challenging than implementing a distributed query system for distributed data. Aloha, Rich Richard L. Pyle, PhD Database Coordinator for Natural Sciences and Associate Zoologist in Ichthyology Department of Natural Sciences, Bishop Museum 1525 Bernice St., Honolulu, HI 96817 Ph: (808)848-4115, Fax: (808)847-8252 email: deepreef(a)bishopmuseum.org http://hbs.bishopmuseum.org/staff/pylerichard.html

1 0

[tdwg-tapir] Output model definition and mixed name spaces.
by Roger Hyam 02 Jan '06

02 Jan '06

Hi Everyone, In the few days between Christmas and New Year I am doing some thinking about architecture and the general problem of validating (controlling) data alongside using a more RDF based approach. This is an open world / closed world problem really but relates to Tapir as follows. Suppose I want to specify an output model like this: <?xml version="1.0"?> <rdf:RDF xmlns:rdf = "http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#" xmlns:vCard = "http://www.w3.org/2001/vcard-rdf/3.0#"> <rdf:Description> <dc:creator> <rdf:Description rdf:about = "http://qqqfoo.com/staff/corky" > <rdfs:label> Corky Crystal </rdfs:label> <vCard:FN> Corky Crystal </vCard:FN> <vCard:N rdf:parseType="Resource"> <vCard:Family> Crystal </vCard:Family> <vCard:Given> Corky </vCard:Given> <vCard:Other> Jacky </vCard:Other> <vCard:Prefix> Dr </vCard:Prefix> </vCard:N> <vCard:BDAY> 1980-01-01 </vCard:BDAY> </rdf:Description> </dc:creator> </rdf:Description> </rdf:RDF> This is taken from: http://dublincore.org/documents/dcq-rdf-xml/#sec3 and is a good example or using a mix of 'other' vocabularies to describe something. DublinCore and vCard are pretty standard and should really not be re-invented. I can't see how to define this structure in a simple XML Schema. It only allows a single target namespace so one needs to have at least 4 XSD documents (1 + 3 imported) to define this tightly. Here is an example of how to do this kind of thing: http://www-128.ibm.com/developerworks/xml/library/x-tipschnm.html We have the additional problem that there aren't actual public XML Schema definitions of these things so one would have to make up schemas defining the elements used. It all gets quite horrible. Does the subset of XML Schema in Tapir support imports? Tapir has the intention of combining separate 'concepts' (which implies different namespaces) into a single output model but do the concepts all have to be mapped into a single namespace to make the thing workable? I may be missing something obvious with XML Schema or Tapir (or both) in which case I would be grateful if some one could set me right. I will blame it all on an excess of rich food and red wine over the past few days. Many thanks for your thoughts. Happy New Year, Roger -- ------------------------------------- Roger Hyam Technical Architect Taxonomic Databases Working Group ------------------------------------- http://www.tdwg.org roger(a)tdwg.org +44 1578 722782 -------------------------------------

3 3

Re: LSID manuscript
by Donald Hobern 02 Jan '06

02 Jan '06

Rod, Thanks for this. It's a really helpful overview of the integration of LSIDs and RDF (and indeed of GUIDs and metadata in general). Donald --------------------------------------------------------------- Donald Hobern (dhobern(a)gbif.org) Programme Officer for Data Access and Database Interoperability Global Biodiversity Information Facility Secretariat Universitetsparken 15, DK-2100 Copenhagen, Denmark Tel: +45-35321483 Mobile: +45-28751483 Fax: +45-35321480 --------------------------------------------------------------- -----Original Message----- From: Taxonomic Databases Working Group GUID Project [mailto:TDWG-GUID@LISTSERV.NHM.KU.EDU] On Behalf Of Roderic Page Sent: 31 December 2005 16:29 To: TDWG-GUID(a)LISTSERV.NHM.KU.EDU Subject: LSID manuscript I've finally managed to put down some ideas on LSIDs, metadata, and taxonomic names in the form of a manuscript. It's a bit rough, but I thought members of this list might find it of interest. You can grab a PDF here: http://darwin.zoology.gla.ac.uk/~rpage/lsid/examples/lsid.pdf The web site http://darwin.zoology.gla.ac.uk/~rpage/lsid/examples has links to some example RDF files that are mentioned in the manuscript (where they are trimmed to save space). I'd welcome any comments. Regards Rod ------------------------------------------------------------------------ ---------------------------------------- Professor Roderic D. M. Page Editor, Systematic Biology DEEB, IBLS Graham Kerr Building University of Glasgow Glasgow G12 8QP United Kingdom Phone: +44 141 330 4778 Fax: +44 141 330 2792 email: r.page(a)bio.gla.ac.uk web: http://taxonomy.zoology.gla.ac.uk/rod/rod.html reprints: http://taxonomy.zoology.gla.ac.uk/rod/pubs.html Subscribe to Systematic Biology through the Society of Systematic Biologists Website: http://systematicbiology.org Search for taxon names at http://darwin.zoology.gla.ac.uk/~rpage/portal/ Find out what we know about a species at http://ispecies.org

1 0