Dear all,

In my (as a non taxonomist) point of view, the distributed GBIF model is not the real problem but the protocol.

DiGIR etc try to offer 'quasi SQL' query possibilities (The 'SOAP' approach) in a distributed environment. However, even GBIF apparently does not use the protocol as it was intended, it uses a 'Harvesting approach' and offers access to the metadata via its cache (please correct me if Im wrong) probably because of some provider problems.

As I mentioned in an earlier mesage,we favor here the harvesting approach. Especially in a highly heterogenous metaddata store which offers eg. oceonographic, biogeographic, taxonomic and literature data the protocol needs to be almost independent from the metadata. We use OAI to harvest most of these third party data sources. Depending on the portal purpose we (will) offer the appropriate query possibilities.

It would be very interesting to me whether OAI ever was discussed in the TDWG or for use in GBIF?

best regards, Robert

-----Ursprungliche Nachricht----- Von: Taxonomic Databases Working Group GUID Project [mailto:TDWG-GUID@LISTSERV.NHM.KU.EDU]Im Auftrag von Roderic Page Gesendet: Mittwoch, 4. Januar 2006 13:51 An: TDWG-GUID@LISTSERV.NHM.KU.EDU Betreff: Re: RDF query and inference in a distributed environment

I wonder whether at some point we need to think carefully about why we have a distributed model in the first place. Is it the best choice? Did we chose it, or has it emerged primarily because data providers want/need to keep control over "their" data?

In the context of bioinformatics, I'm not aware of any large scale distributed environments that actually work (probably ignorance on my part). The largest databases (GenBank, PubMed, EMBL, DDBJ) have all the data stored centrally, and mirror at least some of it (e.g., sequences are mirrored between GenBank, EMBL, and DDBJ). Hence, issues of synchronisation are largely limited to these big databases, and hence manageable.

My sense is that there is a lot of computer science research on federated databases, but few actual large-scale systems that people make regular use of.

What is present in bioinformatics are large numbers of "value added" databases that take GenBank, PubMed, etc. and do neat things with them. This is possible because you can download the entire database. Each one of these value added databases does need to deal with the issue of what happens when GenBank (say) changes, but because GenBank has well defined releases, essentially they can grab a new copy of the data, update their local copy, and regenerate their database.

Having access to all the data makes all kinds of things possible which are harder to do if the data is distributed. I'd argue that part of the success of bioinformatics is because of data availability.

Hence, my own view (at least today) is:

1. Individual data providers manage their own data, and also make

available their data in the following ways: i) provide GUIDs and metadata (e.g., LSIDs) ii) provide a basic, standard search web service iii) provide their own web interface iv) periodically provide complete dump of data

2. GBIF (or some equivalent) takes on the job of harvesting all

providers, building a warehouse, and making that data available through i) web interface ii) web services iii) complete data dump

3. Researchers can decide how best to make use of this data. They may

wish to get a complete "GBIF" dump and install that locally, or query GBIF, or they may wish to query the individual providers for the most up to date information, or some mixture of this.

Issues of synchronisation are dealt with by GBIF and its providers, which I think essentially amounts to having versioning and release numbers (but I'm probably being naive).

Probably 1iv and 3iii are going to cause some issues, and this is off topic, but if bioinformatics is anything to go by, if we don't make our data available in bulk we are tying our own hands. However, this is obviously something each individual provider will have to decide upon themselves.

My other feeling is that from the point of view of end users (and I class myself as one) the real game will be services, especially search (think "Google Life"). And my feeling is that this won't work if queries are done in a distributed fashion -- the Web is supremely distributed, but Google doesn't query the Web, it queries its local copy.

In summary, I think the issue raised by Rich is important, but is one to be addressed by whoever takes on the task of assembling a data warehouse from the individual providers. Of course, once providers make their data available, anybody can do this...

Regards

Rod

---

---

Professor Roderic D. M. Page Editor, Systematic Biology DEEB, IBLS Graham Kerr Building University of Glasgow Glasgow G12 8QP United Kingdom

Phone: +44 141 330 4778 Fax: +44 141 330 2792 email: r.page@bio.gla.ac.uk web: http://taxonomy.zoology.gla.ac.uk/rod/rod.html reprints: http://taxonomy.zoology.gla.ac.uk/rod/pubs.html

Subscribe to Systematic Biology through the Society of Systematic Biologists Website: http://systematicbiology.org Search for taxon names at http://darwin.zoology.gla.ac.uk/~rpage/portal/ Find out what we know about a species at http://ispecies.org