March 2006 - tdwg-tag - lists.tdwg.org

Call for Participation: LSID Working Groups
by Ricardo Scachetti Pereira 07 Mar '06

07 Mar '06

TDWG/GBIF Call for Participation: Life Sciences Identifiers (LSID) Working Groups --------------------------------------------------------------------------- Background ---------- The First Workshop on Globally Unique Identifiers for Biodiversity Informatics (GUID-1) was held last month at NESCent (www.nescent.org) in Durham, North Carolina, USA. The workshop was organized by the Taxonomic Databases Working Group (http://www.tdwg.org/) and the Global Biodiversity Information Facility (http://www.gbif.org/). GUID-1 recommended the use of Life Sciences Identifiers (LSID) in biodiversity informatics and established two Working Groups to address outstanding questions (see http://wiki.gbif.org/guidwiki/). Answers to those questions will be reported to a second GUID meeting (GUID-2) to be hosted by e-Science Institute (eSI) in Edinburgh scheduled for June 10-12 2006. Participation ------------- We have reached a significant point by establishing the infrastructure to support widespread use of LSIDs in biodiversity informatics. We are seeking anyone who is interested in participating in the two Working Groups. Please send an expression of interest in leading or contributing to any of the proposed tasks on the GUID wiki to guids(a)gbif.org. The tasks are listed on the following web pages: http://wiki.gbif.org/guidwiki/wikka.php?wakka=InfrastructureWorkingGroup and http://wiki.gbif.org/guidwiki/wikka.php?wakka=PrototypingWorkingGroup If you can lead a task team, please edit the respective wiki page, adding your name as task coordinator and start to outline the task. If you want to be involved but not lead a task, contact the task leader if there is one, or edit the respective wiki page by adding your name to the task team and adding your contribution in the appropriate space. GUID-2 ------ The second GUID meeting will be supported by the TDWG Infrastructure Project funded by the Gordon and Betty Moore Foundation. Attendance at GUID-2 will be by invitation, based on contributions against the tasks. GUID-2 will focus on working sessions and attendance will be limited. Priority for support will be given to workgroup task leaders and others actively participating on task developments. Timeline -------- 17/Mar/2006: Definition of task leaders. Tasks not claimed will be pruned, reassigned or set aside for later consideration. 24/Mar/2006: Definition of task teams, task outlines, and work plan. 31/Mar/2006: Definition of GUID-2 participants based on task assignments. GBIF/TDWG will send formal invitations to participants. Mar/Apr/May/2006: Task development and discussion of outstanding issues, following work plan. May/2006: Finalization of prototypes, summary of discussions, draft of recommendations for each task. Preparation to present summary at GUID-2 Workshop. June/2006 - GUID-2 Workshop. Contacts -------- For further information, please send messages to guids(a)gbif.org, or contact any of the following- * Lee Belbin, Australia - Project Manager (lee(a)tdwg.org) * Donald Hobern, Denmark - Programmer Officer for Data Access and Database Interoperability (dhobern(a)gbif.org) * Ricardo Pereira, Brazil - Software Engineer (ricardo(a)tdwg.org) * Roger Hyam, United Kingdom - Technical Architect (roger(a)tdwg.org)

1 0

Re: [Tdwg-tag] Why should data providers supply search and query services?
by "Döring, Markus" 06 Mar '06

06 Mar '06

Roger, I really think its time to discuss the issue of indexers and searchable providers. And I have to say I mostly do agree with your analysis, although consequently this would mean to abandon our DiGIR, BioCASE and TAPIR protocols. Instead we could go for OAI or any other similar standard. Or we could create our own minimal TDWG sync protocol to retrieve lists of changed records. Historically we created search protocols to avoid central caches; mainly not to scare providers, to convince them to publish their data so that they are keeping control of it. But that argument is probably gone by now. Although there is the need to discuss these things in the architecture group, we should be careful that TDWG will not become a synonym for GBIF. How people are using TDWG standards should be left open to some degree. Someone might want to set up searchable providers, others not. >>>From my feeling I guess this might actually be different depending on the kind of data (objects) they are exchanging. As it looks to me we currently cannot create a reliable and fast fully distributed system. So the initial DiGIR and Co dream has somehow failed. For a small number of providers with a good server infrastructure and relatively small data its a different thing though. But in general for many if not all applications we will have to have some kind of local cache. A huge warehouse keeping all our data is quite challenging though. I doubt we could fill a single system with all our thousands of concepts and 100 millions of records. In RDF we will end up with several billions of triples! But as soon as we start selecting subsets of relevant attributes (concepts), interpret incoming data to harmonize it or to remove "obvious" errors, we render the cache useless for other applications. Updating intervalls and reliability are also issues that freak out clients. So in the worst case we might end up having a seperate cache for every different application. So the heavy burden of indexing becomes a problem for most of our clients! I am not suggesting that this is wrong, but I dont think the indexing problem will only touch a few. Searchable providers allow us to do tests in advance and create ad-hoc networks that dont need central infrastructures. Many providers are currently being convinced to publish their data just because we bundle local portals with the software. They immediately have a search interface on the web which is also great for local networks especially in smaller institutions with little IT resources. This is only possible via the generic search interface we are providing with DiGIR, BioCASE or TAPIR. >>>From a pure technical point I would probably argue for LSIDs, RDF and OAI. But if you think about the consequences, it would mean starting from scratch. Its a huge change and I cant think of much that would stay the same. I am afraid that we simply cant cope with that and by the time we are close to having a productive system it turns out there are better ways of doing this. Maybe distributed XQuery is simple to use by then? or a SPARQL server in front of our providers is easy to set up and fast to use? Or GRID is finally taking over all of us! who knows. So for sure we have to address all the issues in a wider audience. Many people outside the TAG or GUID group are not aware of the proposed changes and would be really suprised to know about the consequences. Still I hope we can make a transition to RDF at some point. Maybe the solution lies in the combinations of both technologies for some time? Integrating TAPIR, LSID resolution and OAI(?) into DiGIR2, PyWrapper and the GBIF Indexing would smooth transition. I am sure there's more to say, but the weekend is approaching and I think I consumed too much of all your attention already. Markus -----Ursprüngliche Nachricht----- Von: Tdwg-tag-bounces(a)lists.tdwg.org [mailto:Tdwg-tag-bounces@lists.tdwg.org] Im Auftrag von Roger Hyam Gesendet: Freitag, 3. März 2006 15:44 An: Bob Morris Cc: Tdwg-tag(a)lists.tdwg.org Betreff: Re: [Tdwg-tag] Why should data providers supply search and query services? Bob, Great points on provenance and maintaining standards whether they are moving between provider, aggregator, indexer, client or whatever. GUIDs should be a help in tracking provenance but we do need policies on what aggregators can do with objects... I don't think my suggestions preclude anything. Perhaps they can be summed up as a suggestion that providers, indexers and search and query services should be modeled as separate actors within the architecture. Some software instances may play the roles of more than one of these actors but this is not compulsory . Roger Bob Morris wrote: On 3/3/06, Roger Hyam <roger(a)tdwg.org> wrote: Bob Morris wrote: Umm...there is a distinguishable class of data consumers, namely applications, and so a distinguishable constituency whose burden is relevant, namely application writers. Some applications may well be motivated to query providers directly for a number of reasons, including: * the data indexers currency policies may be unsuitable This equally applies to data providers. They may not index data in a way the consumer requires. It may lag behind their own live data set etc. I agree completely on this and your other dittos. It's typically hard to figure out whether something is an aggregator or an originator. This is the oft-discussed issue of "data provenance" which is quite difficult to establish on a per-record data. In the (defunct?) UBIF schema there is a weak attempt to record how, or at least if, a record evolved from its originator. Furthermore, the history of that evolution, were it understood (by a machine!) could prove quite useful to an application, which may well find it interesting to incorporate the wisdom of intermediaries and find some of them provide a better view of a given record than do others, possibly even including the originator. As a simple example, It could be quite convenient if an intermediiary that by some clever processing could establish that some datum in a record is inconsistent with some other in the same record and could record that fact in its forwarding metadata. Really, my vision here is machines as scholars. I don't suggest TDWG should attempt to accomplish that. I merely say that if that is one's vision, then one buries fewer difficiult to extract assumptions in the modeling. I think this is the real point of my arguments: how to recognize all the "gotchas" in one's models and make sure they are acknowledged enough that others can deal with them. ["Gotcha" is an Americanism(?) contracted from "I got you!" typically uttered to the victim of a practical joke who has been successfully blind-sided]. [As an aside, I note that the much vaunted data-information-knowledge pyramid is actually cited as data-information-knowledge-wisdom by some authors. Scientists too often stop at "knowledge" because "wisdom" seems too hard to define and perhaps a little too uncomfortable to assert about oneself. ] * the data indexers may aggregate in undesirable ways [the present model seems to be that indexer==portal, but I doubt that is general] Ditto from point above. Data suppliers may index in undesirable ways plus they might index heterogeneously - each supplier may be undesirable in different ways - which would be a really big headache. Is there anything to say this will cause less of a burden when spread across many providers rather than few indexers? If a thematic indexer doesn't do what is required then it may be possible to get something changed. If 50 suppliers don't index something correctly then it will no doubt take years to get any changes affected - especially if they are all doing it wrong differently. This might also be addressed by good provenance trails in the data. [Iterate this sentiment for all your dittos...] * the data indexers may index too promiscuously or not promiscuously enough for the application's taste [this might be a non-issue if there were a way for a machine to understand what exactly the indexing strategy is and perhaps how to induce the indexer to alter it, but that sounds hard] Again ditto. If providers are also indexers then any criticism of problems with indexing has to apply to the suppliers but is magnified by the number of suppliers. * portals, and maybe indexers---indeed, any processor of the data---can intentionally or inadvertantly hide assumptions about how the data will be used, making it unsuited for uses that don't meet these assumptions. Put another way, it is probably difficult to insure that a machine-enforceable contract is possible between aggregators and applications that assures the application that records obtained from the aggregator or identical to those available from the provider. I think it is even a deep problem to have machine-understandable "fitness for use" metadata that would allow a machine to understand what fitness contract the aggregator is actually offering. I would assume that the aggregator is assembling metadata (in the sense of things that can be searched on) rather than actual data. The aggregator/indexer is really only providing a GUID discovery service. The consumer can always retrieve the original objects from the data supplier. The aggregator/indexer is only providing a match making service. As to "only", I agree for indexers but doubt it for aggregators. Sometimes. In general it should never be harder to query providers than aggregators, especially if it is difficult for a machine to understand what, if any, point of view the aggregator has imposed on the view they offer of the aggregated data. I don't believe this follows from your points above: I frequently go to websites and can't find what I want so I go to Google and do a search restricting its scope to just that site. Indeed Google provide this as a service - just embed a search box on your site that passes the right parameters. In this situation it is definitely easier to query the aggregator than the supplier. Indeed many sites don't bother with providing search services other than Google (which is the point I make precisely). The alternative is that every tin-pot website has to have an implementation of the Google search algorithm and indexes within it. (I appreciate that this is a human example but it translates to a machine world. A data provider's metadata could easily provide the location of web services to query it that are not actually part of the provider itself. Indeed it could offer a list of services. A neat place to do this would be in the WSDL returned by a LSID Authority.) Good point. Google deserves thought. If it is an aggregator other than trivially, it is certainly one with a point of view, a hint of which can be seen in their cached pages, where they helpfully add to the data by highlighting the search terms. Who asked for that? Not me. But I don't seem to be offered a choice about it. Conversely, someone who desires to take advantage of Google's wisdom in this regard may actually find their view more useful than the originator's. Indeed, for me it frequent that I go to the original page and then am frustrated by the weak Firefox search facility when I try to figure out where in the original I should be looking. But if I use the Google cache, I may be at the mercy of their currency policies. This frequently makes it not so useful in searching for things in archived poorly threaded archives such as email archives---if the discussion is so old that the Google cache is complete it is sometimes the case that the answer is in the originator but hard to find, yet not in the cache where it would be easy to find. People are no doubt tired of hearing this from me, but my position is always that modeling data consumers as humans is dangerously constricting. Humans are too smart and readily deal with lots of violations of the principle of least amazement, whereas machines don't. In point of fact, except for those on paper, stone, clay tablets and the like, there is no such thing as a database accessed by a human. They all have software between the human and the data provision service. From this I conclude that in your trinity below, reduction of the burden on humans actually falls to the applications, and so I think TAGs requirement is to reduce the burden on application writers (including those of TDWG itself, but also all others in the world) in their quest to reduce the burden on human data consumers. My intuition is that this will lead to a different analysis than thinking about humans as consumers, but at the moment I have no specific examples to offer. I think this is a really good point and will take it forward. I hope to start the TAG meeting with a discussion of Actors within our domain and will attempt to differentiate client-human from client-machine within this. I often muse upon the fact that the UML Actor symbol doesn't distinguish human from non-human actors. There are good and bad aspects of that. Good when you are modeling a software system. Bad when there are actually humans who can push the buttons. [Or maybe it's really good if you are constantly aware that humans behave unexpectedly. Keeping that in mind is the real point about my "forbidden questions"]. A little more is interspersed below. On 3/1/06, Roger Hyam <roger(a)tdwg.org> wrote: This is a little more of a controversial question that has been suggested: "Why should data providers supply search and query services?" * We have many potential data providers (potentially every collection and institution). * We have many potential data consumers (potentially every researcher with a laptop). * We have a few potential data indexers (GBIF, ORBIS , etc + others to come). The implementation burden should therefore be: * Light for the providers - who's role is to conserve data and physical objects. * Light for the consumer - who's role is to do research not mess with data handling. * Heavy for the indexers - who's core business is making the data accessible. Data providers should give the objects they curate GUIDs. This is important because it stamps their ownership (and responsibility) on that piece of data. They then need to run an LSID service that serves the (meta)data for the objects they own. There work should stop at this point! They should not have to implement search and query services. They should not anticipate what people will require by way of data access - that is a separate function. Data consumers should be able to access indexing services that pool information from multiple data providers. They should not have to run federated queries across multiple data providers or have to discover providers as this is complex and difficult (though they may want to browse round data providers like they would browse links on web pages). Once they have retrieved the GUIDs of the objects they are interested in from the indexers they may want to call the data providers for more detailed information. Data indexers should crawl the data exposed by the providers and index them in thematic ways. e.g. provide geographic or taxon focused services. This is a complex job as it involves doing clever, innovative things with data and optimization of searches etc. Currently we are trying to make every data provider support searching and querying when the consumers aren't really interested in querying or searching individual providers - they want to search thematically across providers. Restated, this sentence may fall in my class of questions forbidden to software architects, namely that class of questions that begin with the words "Why would anybody ever want to ..." I should restate it "What is the use case that indicates the system should support this behavior?" If a big data provider wants to provide search and query then they can set themselves up as both a provider and an indexer - which is more or less what everyone is forced to do now - but the functions are separate. Data providers would have to implement a little more than just an LSID resolver services for this to work. They would need to provide a single web service method (URL call) that allowed indexers to get lists of LSIDs they hold that have had their (meta)data modified since a certain date but this would be a relatively simple thing compared with providing arbitrary query facilities. I believe (though I haven't done a thorough analysis of log data ) that this is more or less the situation now. Data providers implement complete DiGIR or BioCASE protocols but are only queried in a limited way by portal engines. Consumers go directly to portals for their data discovery. So why implement full search and query at the data provider nodes of the network (possibly the hardest thing we have to do) when it may not be used? This may be controversial. What do you think? I'm not sure about controversial, but I am pretty sure that what you are pointing at is a warehouse model. I don't know if I am prepared to agree that all possible present and future concerns of TDWG can be answered by data warehouses. In particular, if you analyse log data of a warehouse, it won't be too surprising if the conclusion is that users are behaving as though they mainly need a warehouse. [To data consumers a warehouse and a portal are indistinguishable. I think.] This is why I use the term 'indexer' rather than aggregator. The analogy with web search engines is a good one. Basically we have to implement aggregated-indexes for key data (although federated searching by crawling all the providers is theoretically possible if you are not in a hurry) the question I raise is whether we also need to implement querying in every provider. Maybe not. What would alarm me though, is if we do something that precludes it or even makes it hard. I could grudgingly live with a position that TDWG's service function definitions are all about aggregation. But the data exchange standards had better not distinguish aggregators from originators from transformers except for providing those actors with the ability to identify their role and point of view. Bob Morris Roger -- ------------------------------------- Roger Hyam Technical Architect Taxonomic Databases Working Group ------------------------------------- http://www.tdwg.org <http://www.tdwg.org> roger(a)tdwg.org +44 1578 722782 ------------------------------------- _______________________________________________ Tdwg-tag mailing list Tdwg-tag(a)lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-tag_lists.tdwg.org -- ------------------------------------- Roger Hyam Technical Architect Taxonomic Databases Working Group ------------------------------------- http://www.tdwg.org roger(a)tdwg.org +44 1578 722782 -------------------------------------

5 8

Re: [tdwg-tapir] conceptual binding
by "Döring, Markus" 06 Mar '06

06 Mar '06

Personally I wouldnt wanna implement anything more than qualified paths to be honest. Its definitely much more work to deal with predicates, relative paths or unknown nesting levels ala //. So couldnt we just use this pattern for validation? (/([a-zA-Z0-9_-]+:)?[a-zA-Z0-9_\-]+)+(@[a-zA-Z0-9_\-]+)? Strings are fine as well, but Id prefer validation in exchange for the lack of extensibility. markus -----Ursprüngliche Nachricht----- Von: tdwg-tapir-bounces(a)lists.tdwg.org [mailto:tdwg-tapir-bounces@lists.tdwg.org] Im Auftrag von Renato De Giovanni Gesendet: Freitag, 3. März 2006 02:48 An: tdwg-tapir(a)lists.tdwg.org Betreff: Re: [tdwg-tapir] conceptual binding Hi all, I would be reluctant to force provider implementations to understand more complex XPath patterns without having a clear demand from our community. We obviously need optional namespace prefixes in the pattern since it's already clear that output models may easily mix different namespaces. But I don't know if it's wise to go beyond that. Adopting the full Location Paths specification seem an overkill, and even the WFS predicates look strange in our context. I did find some examples about using them in WFS: http://mail.opengeospatial.org/pipermail/wfs-dev/2004- August/000189.html But it's important to note that WFS uses XPath in a different context. In TAPIR we just use them in the output model mapping section and also to select partial nodes in search templates. TAPIR filters don't reference XML nodes, they reference concepts (which have a completely different type). Back to XPath predicates, maybe someone could want to get back only the "n" identification of an ABCD unit, for instance. So I can imagine situations when <partial> could make use of predicates, but for me the cost-benefit still doesn't seem to justify the additional complexity. And it's hard to believe that someone could really want to map only the "n" occurrence of a specific XML element in an output model. But maybe I'm wrong... Javier's suggestion to remove the SimpleXPathType restriction and only describe in the specs the minimum patterns to be understood by providers seems reasonable. We could even allow provider implementations to understand more complex patterns if they want. Regards, -- Renato On 3 Mar 2006 at 13:04, Javier de la Torre wrote: > Hi Markus, > > No. I was just thinking in adding something like Location Paths > (http://www.w3.org/TR/xpath#location-paths) or we could make use of > what the OGC defines in WFS: > >From WFS specifications (1.0.0 and 1.1.0) > """ > This specification does not require a WFS implementation to support > the full XPath language. In order to keep the implementation entry > cost as low as possible, this specification mandates that a WFS > implementation must support the following subset of the XPath > language: > 1. A WFS implementation must support abbreviated relative location paths. > 2. Relative location paths are composed of one or more steps separated > by the path separator '/'. > 3. The first step of a relative location path may correspond to the > root element of the feature property being referenced or to the root > element of the feature type with the next step corresponding to the > root element of the feature property being referenced. > 4. Each subsequent step in the path must be composed of the > abbreviated form of the child:: axis specifier and the name of the > feature property encoded as the principal node type of element. The abbreviated form of the child:: > axis specifier > is to simply omit the specifier from the location step. > 5. Each step in the path may optionally contain a predicate composed > of the predicate delimiters '[' and ']' and a number indicating which > child of the context node is to be selected. This allows feature > properties that may be repeated to be specifically referenced. > 6. The final step in a path may optionally be composed of the > abbreviated form of the attribute:: axis specifier, '@', and the name > of a feature property encoded as the principal node type of attribute. > """ > In the WFS schema there is a type called XlinkPropertyName to care of > this, but I don't know if this is suitable for us... > > The other possibility is to set it to xsd:string and in the > documentation explain that the implementation is free to define up to > which level they want to support Xpath there and maybe define the > minimum as we had before. > > Javier. > > On 3/3/06, "Döring, Markus" <m.doering(a)bgbm.org> wrote: > > Well, what do you consider a proper xpath type? > > I think we can add some basic things to our simple pattern. But are you also thinking of adding different axes, functions and all this? Things I could see to be allowed are: > > > > (pre:)qname Selects all child nodes named like this (optionally qualified with prefixes) > > / Selects from the root node > > // Selects nodes in the document from the current node that match the selection no matter where they are > > .. Selects the parent of the current node > > @ Selects attributes > > Predicates: > > [i] Select ith node > > [last()-1] Select second-last node > > [position()<3] Select all nodes before the third > > > > > > > > But that already means we could get a list of matching nodes from the xpath. So a simple mapping becomes a mapping to a list of nodes - essentially multiple single mappings. Gregor was actually looking for this in the recursive SDD. > > > > Is this overkill already? > > Markus > > > > > > -----Ursprüngliche Nachricht----- > > Von: tdwg-tapir-bounces(a)lists.tdwg.org > > [mailto:tdwg-tapir-bounces@lists.tdwg.org] Im Auftrag von Javier > > privat > > Gesendet: Donnerstag, 2. März 2006 19:48 > > An: Renato De Giovanni > > Cc: tdwg-tapir(a)lists.tdwg.org > > Betreff: Re: [tdwg-tapir] conceptual binding > > > > Hi, > > > > > > Now looking at the "simpleXPathType" (used by the node's path > > > attribute) it seems that the string restriction does not allow for > > > namespace prefixes. So I guess this is something that needs to be > > > fixed in the schema. > > > > Actually the problem with the simpleXpathType arise when I did the example outputModel, it does not validate against the schema because I used ns prefixes. > > > > Maybe my sentence was incorrect, I wanted to say "The change in the mapping section is only in allowing (forcing?) qualifies paths in the node element" > > > > Actually I think pywrapper would not complain of the lack of ns prefixes there (probably the other way around :D ), but in the unlikely event that the ns resolve ambiguities then we are in troubles. > > > > And actually I don't know why this path can not be an xpath. > > pywrapper and Digir will not be able to undertand complicate xpaths, but maybe other implementations can, and I don't see what is the problem if the provider arise and error like "Sorry xpath in the mapping section of the outputModel not understood". > > > > Imagine a TAPIR implementation that allows you to do a mapping to the second element </a>using xpath expressions. We would be able to do mappings to structures like: > > > > <a> > > > > content here > > </a> > > > > So here is my proposal: change the simpleXPathType to a proper > > XpathType. > > > > Javier. _______________________________________________ tdwg-tapir mailing list tdwg-tapir(a)lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-tapir_lists.tdwg.org

2 1

Technology/Approach to core Vocabulary/Ontology
by Roger Hyam 04 Mar '06

04 Mar '06

Hi All, I believe we agree that we need a core vocabulary/ontology and that this is limited in scope and under the remit of the TAG for the time being. So my first question is: *How do we develop a core vocabulary/ontology in a distributed, collaborative way?* Renato pointed out the way CIDOC is modeled which is similar (but more complex) to the way DublinCore is presented. Should we just start with a wiki page and build it from their or is there a more sophisticated tool we should use? Future use cases might include linking existing schemas to the core ontology and developing domain specific ontologies. My second question is: *Can we separate the definition of terms from the ontologies? * An example might make this clearer. We may be able to define TaxonName, GenusName and SpecificEpithet as terms and get general agreement on their meaning. This is separate from defining an ontology that stipulates TaxonName can only have a SpecificEpithet if it has a GenusName. One community may choose to use an ontology (or schema) to enforce this behaviour another might not but it doesn't change the meaning of the terms. DublinCore is like this. It defines a set of terms (as RDF properties in one form) that can be bolted together any way a user chooses but then the PRISM standard enforces their use in a certain way. Could decoupling of core terms from ontologies be a way forward? I am willing to stick my neck out and put the first list of objects/terms together but we then need to discuss use cases and modify any initial list. I believe this will then feed into protocols and serializations etc.. Once again I am grateful for your thoughts, Roger -- ------------------------------------- Roger Hyam Technical Architect Taxonomic Databases Working Group ------------------------------------- http://www.tdwg.org roger(a)tdwg.org +44 1578 722782 -------------------------------------

2 1

implementation of CIDOC-CRM
by Karl-Heinz Lampe 03 Mar '06

03 Mar '06

Dear All, Concerning the question of implementations of CIDOC CRM there might be a misunderstanding. There are none because the CIDOC-CRM is a core ontology which serves as (1) an intellectual guide to create schemata, formats, profiles and (2) as a language for analysis of existing sources and models for data integration (mapping). That means in this context that - on addition to CIDOC-CRM - there is a need of a protocol (such as the OAI [open archive initiative]) serving as means of transport. NB the CIDOC CRM has been developed because of already existing different schematas to make them commonly understandable. It was primarily not developed to create new schemata (of course one can use it for that purpose as a best practise guide). That is all for the time being. Best Karl Ps: I am not in for the next two weeks. ________________________________________________ Dr. Karl-Heinz Lampe Head of Biodiversity Informatics, Curator Zoologisches Forschungsmuseum Alexander Koenig (ZFMK) Adenauerallee 160 53113 Bonn Tel: ++49 (0)228 9122290 Fax: ++49 (0)228 9122291 http://www.zfmk.de http://www. biodat.de e-mail: k.lampe.zfmk(a)uni-bonn.de ________________________________________________ -----Original Message----- From: On Behalf Of Roger Hyam Sent: Thursday, February 23, 2006 3:09 PM To: Renato De Giovanni Cc: Tdwg-tag(a)lists.tdwg.org Subject: Re: [Tdwg-tag] Object Model / Ontology Management - how we kick it off. Hi Renato, Thanks for that contribution. I have had CIDOC on my list since the St Petersburg meeting last year. We definitely need to address how we make use of it or integrate with it. My worry is looking for implementations that use it. I am not aware of organisations sharing data on the basis of schemas derived from the CIDOC model - this is worrying as the ontology has been under development for 10 years - it may also be that I am ignorant. Does anyone on the list have practical experience of using CIDOC? Thanks, Roger Renato De Giovanni wrote: Roger, I think I agree with most of your points (also from previous messages). Concerning the representation independent object model, I would suggest the same approach taken by CIDOC CRM: <http://cidoc.ics.forth.gr/docs/cidoc_crm_version_4.2.pdf> http://cidoc.ics.forth.gr/docs/cidoc_crm_version_4.2.pdf It's not easy to find such clear and well documented modelling work. Although at a first glance it could fall into the "ontology-at-the- level-of-laws-of-physics" category, I won't make that judgment because CIDOC's scope is definitely broader than ours. Anyway, what I'm suggesting is to use the same approach and the same kind of documentation. Using and extending CIDOC is a completely different thing - probably interesting (I think), but something that could even be evaluated and addressed at another stage. Regards, -- Renato On 22 Feb 2006 at 15:50, Roger Hyam wrote: Hi All, It is generally agreed that we need an representation independent object model or ontology of some kind. I would like to put together a list of the things that need to be agreed or investigated in order to do this. Firstly the things I believe we can all agree on (stop me if I am wrong). * It should be representation independent (i.e. we should be able to move it between 'languages' UML, OWL, BNF etc). * It should be dynamic (i.e. capable of evolving through time). * It should be polymorphic. This is a result of it being dynamic. There will, at a minimum, be multiple version of any one part of the model when new version are introduced. * It should NOT attempt to be omniscient i.e. it will not cover everything in our domain, only the parts that need to be communicated. * It will be managed in a distributed fashion. Different teams will take responsibility for different parts of it. My first Question is: Does the centralization of the ontology need to go beyond a small shared vocabulary of terms or base classes? I envisage this ontology containing things like Collection, Specimen, TaxonConcept, TaxonName but not defining the detailed structure of these objects. It would contain a maximum of a few 10's of objects and properties. TDWG subgroups would be responsible for building ontologies that extend these base objects but that generally didn't refer to each other - only to the core. If this is true then I think the definition of the top level object falls within the remit of the TAG ( in consultation with others). If this is not a valid way forward what are the alternatives? Are their questions we should ask before this one? Once again I'd be grateful for your thoughts. Roger _______________________________________________ Tdwg-tag mailing list <mailto:Tdwg-tag@lists.tdwg.org> Tdwg-tag(a)lists.tdwg.org <http://lists.tdwg.org/mailman/listinfo/tdwg-tag_lists.tdwg.org> http://lists.tdwg.org/mailman/listinfo/tdwg-tag_lists.tdwg.org -- ------------------------------------- Roger Hyam Technical Architect Taxonomic Databases Working Group ------------------------------------- <http://www.tdwg.org> http://www.tdwg.org <mailto:roger@tdwg.org> roger(a)tdwg.org +44 1578 722782 -------------------------------------

1 0

Re: [tdwg-tapir] conceptual binding
by "Döring, Markus" 03 Mar '06

03 Mar '06

Well, what do you consider a proper xpath type? I think we can add some basic things to our simple pattern. But are you also thinking of adding different axes, functions and all this? Things I could see to be allowed are: (pre:)qname Selects all child nodes named like this (optionally qualified with prefixes) / Selects from the root node // Selects nodes in the document from the current node that match the selection no matter where they are .. Selects the parent of the current node @ Selects attributes Predicates: [i] Select ith node [last()-1] Select second-last node [position()<3] Select all nodes before the third But that already means we could get a list of matching nodes from the xpath. So a simple mapping becomes a mapping to a list of nodes - essentially multiple single mappings. Gregor was actually looking for this in the recursive SDD. Is this overkill already? Markus -----Ursprüngliche Nachricht----- Von: tdwg-tapir-bounces(a)lists.tdwg.org [mailto:tdwg-tapir-bounces@lists.tdwg.org] Im Auftrag von Javier privat Gesendet: Donnerstag, 2. März 2006 19:48 An: Renato De Giovanni Cc: tdwg-tapir(a)lists.tdwg.org Betreff: Re: [tdwg-tapir] conceptual binding Hi, >> > > Now looking at the "simpleXPathType" (used by the node's path > attribute) it seems that the string restriction does not allow for > namespace prefixes. So I guess this is something that needs to be > fixed in the schema. Actually the problem with the simpleXpathType arise when I did the example outputModel, it does not validate against the schema because I used ns prefixes. Maybe my sentence was incorrect, I wanted to say "The change in the mapping section is only in allowing (forcing?) qualifies paths in the node element" Actually I think pywrapper would not complain of the lack of ns prefixes there (probably the other way around :D ), but in the unlikely event that the ns resolve ambiguities then we are in troubles. And actually I don't know why this path can not be an xpath. pywrapper and Digir will not be able to undertand complicate xpaths, but maybe other implementations can, and I don't see what is the problem if the provider arise and error like "Sorry xpath in the mapping section of the outputModel not understood". Imagine a TAPIR implementation that allows you to do a mapping to the second element </a>using xpath expressions. We would be able to do mappings to structures like: <a> content here </a> So here is my proposal: change the simpleXPathType to a proper XpathType. Javier. > > Best Regards, > -- > Renato > > On 2 Mar 2006 at 17:11, Javier de la Torre wrote: > >> Uhmm... >> >> Did I understand this incorrectly? >> The change in the mapping would only be qualifying concepts in the >> path attribute not in the concept ids. >> >> I think we agree on that the concept ids does not have anything to do >> with XML schema or Xpaths no? Is just "by coincide" that we are using >> something "similar". >> >> Javier. >> >> On 3/3/06, Renato De Giovanni <renato(a)cria.org.br> wrote: >>> Hi Markus, >>> >>> This is really good news for the PyWrapper! >>> >>> Now concerning your question, I think we should really try to avoid >>> defining concepts with compound XML schemas. It's a lot cleaner and >>> more elegant using gcp#accession/FullScientificName to map local >>> databases and to use in filters. As you said, the WFS response would >>> just be one of the possible output models making use of the >>> concepts. >>> >>> The real xpaths to WFS elements (including namespaces) would only be >>> used in the output model mapping section: >>> >>> <node >>> path="/wfs:FeatureCollection/gml:featureMember/gcp:accession/ >>> gcp:FullS >>> cientificName"> >>> <concept id="gcp#accession/FullScientificName"/> >>> </node> >>> >>> Best Regards, >>> -- >>> Renato >>> >>> On 28 Feb 2006 at 11:38, "Döring, Markus" wrote: >>> >>>> Hello, >>>> I recently added support for schema imports in pywrapper mainly >>>> to support multiple namespaces in the repsonse. >>>> Me (and Javier) wanted to allow something like this: >>>> >>>> <wfs:FeatureCollection >>>> xmlns:wfs="http://www.opengis.net/wfs" >>>> xmlns:gml="http://www.opengis.net/gml" >>>> xmlns:gcp="http://www.ipgri.org/schemas/gcp_passport_gml/1.0"> >>>> <gml:featureMember> >>>> <gcp:accession fid="accession.12"> >>>> <gcp:GermplasmID>12</gcp:GermplasmID> >>>> <gcp:FAOInstituteCode>ES</gcp:FAOInstituteCode> >>>> <gcp:FullScientificName>Quercus ilex</ >>>> gcp:FullScientificName> >>>> </gcp:accession> >>>> </gml:featureMember> >>>> </wfs:FeatureCollection> >>>> >>>> Here comes my problem: >>>> How do we refer to the FullScientificName concept in TAPIR? >>>> >>>> As a convention for schemas we said to create the fully >>>> qualified concept from the namespace and the simple xpath to the >>>> element. So somethinkg like this: >>>> >>>> http://www.opengis.net/wfs#/FeatureCollection/featureMember/ >>>> accession/FullScientificName >>>> >>>> A real xpath to that element would of course include several >>>> namespaces: >>>> /wfs:FeatureCollection/gml:featureMember/gcp:accession/ >>>> gcp:FullScientificName >>>> >>>> Could it be that we never need to refer to "compound" concepts >>>> anyway? A provider would probably map his db to the GCP schema >>>> alone. The WFS response is only the responded view/data model >>>> and we would still just use gcp#accession/FullScientificName as >>>> the qualified concept in filters. This is also true for our >>>> simple extension schemas I think (EFG for ABCD, all extensions >>>> for DwC). But arent there other cases? >>>> >>>> Thanks, >>>> Markus > > > _______________________________________________ > tdwg-tapir mailing list > tdwg-tapir(a)lists.tdwg.org > http://lists.tdwg.org/mailman/listinfo/tdwg-tapir_lists.tdwg.org _______________________________________________ tdwg-tapir mailing list tdwg-tapir(a)lists.tdwg.org http://lists.tdwg.org/mailman/listinfo/tdwg-tapir_lists.tdwg.org

3 2

[tdwg-tapir] conceptual binding
by "Döring, Markus" 02 Mar '06

02 Mar '06

Hello, I recently added support for schema imports in pywrapper mainly to support multiple namespaces in the repsonse. Me (and Javier) wanted to allow something like this: <wfs:FeatureCollection xmlns:wfs="http://www.opengis.net/wfs" xmlns:gml="http://www.opengis.net/gml" xmlns:gcp="http://www.ipgri.org/schemas/gcp_passport_gml/1.0"> <gml:featureMember> <gcp:accession fid="accession.12"> <gcp:GermplasmID>12</gcp:GermplasmID> <gcp:FAOInstituteCode>ES</gcp:FAOInstituteCode> <gcp:FullScientificName>Quercus ilex</gcp:FullScientificName> </gcp:accession> </gml:featureMember> </wfs:FeatureCollection> Here comes my problem: How do we refer to the FullScientificName concept in TAPIR? As a convention for schemas we said to create the fully qualified concept from the namespace and the simple xpath to the element. So somethinkg like this: http://www.opengis.net/wfs#/FeatureCollection/featureMember/accession/FullS… A real xpath to that element would of course include several namespaces: /wfs:FeatureCollection/gml:featureMember/gcp:accession/gcp:FullScientificName Could it be that we never need to refer to "compound" concepts anyway? A provider would probably map his db to the GCP schema alone. The WFS response is only the responded view/data model and we would still just use gcp#accession/FullScientificName as the qualified concept in filters. This is also true for our simple extension schemas I think (EFG for ABCD, all extensions for DwC). But arent there other cases? Thanks, Markus

4 6

[tdwg-tapir] CNS
by "Döring, Markus" 02 Mar '06

02 Mar '06

hi, is there any particular reason why we are having a primary and secondary concept nameserver only? Cant we allow an unlimited number of servers in an ordered list? the 1st CNS server entry with highest priority in case we encounter any conflicts. markus

2 1

Re: GUID Wiki now highlights URIs starting with lsidres: automatically
by Lee Belbin 02 Mar '06

02 Mar '06

Hi Ricardo That is neat! Cheers, Lee Lee Belbin Manager, TDWG Infrastructure Project Email: lee(a)tdwg.org Phone: +61(0)419 374 133 -----Original Message----- From: Taxonomic Databases Working Group GUID Project [mailto:TDWG-GUID@LISTSERV.NHM.KU.EDU] On Behalf Of Ricardo Scachetti Pereira Sent: Thursday, 2 March 2006 6:34 AM To: TDWG-GUID(a)LISTSERV.NHM.KU.EDU Subject: GUID Wiki now highlights URIs starting with lsidres: automatically Hi all, I've added a feature to the GUID wiki so that it automatically highlights URIs starting with lsidres:. Below are two examples of pages using the feature: http://wiki.gbif.org/guidwiki/wikka.php?wakka=LSIDResolveForTaxonConceptsCoL http://wiki.gbif.org/guidwiki/wikka.php?wakka=LSIDResolverForTaxonNamesUBio This will make it easier to us to create resolvable/actionable links to LSIDs on the wiki. Regards to all, Ricardo

1 0

Object Model / Ontology Management - how we kick it off.
by Roger Hyam 02 Mar '06

02 Mar '06

Hi All, It is generally agreed that we need an representation independent object model or ontology of some kind. I would like to put together a list of the things that need to be agreed or investigated in order to do this. Firstly the things I believe we can all agree on (stop me if I am wrong). 1. It should be representation independent (i.e. we should be able to move it between 'languages' UML, OWL, BNF etc). 2. It should be dynamic (i.e. capable of evolving through time). 3. It should be polymorphic. This is a result of it being dynamic. There will, at a minimum, be multiple version of any one part of the model when new version are introduced. 4. It should NOT attempt to be omniscient i.e. it will not cover everything in our domain, only the parts that need to be communicated. 5. It will be managed in a distributed fashion. Different teams will take responsibility for different parts of it. My first Question is: *Does the centralization of the ontology need to go beyond a small shared vocabulary of terms or base classes?* I envisage this ontology containing things like Collection, Specimen, TaxonConcept, TaxonName but not defining the detailed structure of these objects. It would contain a maximum of a few 10's of objects and properties. TDWG subgroups would be responsible for building ontologies that extend these base objects but that generally didn't refer to each other - only to the core. If this is true then I think the definition of the top level object falls within the remit of the TAG ( in consultation with others). If this is not a valid way forward what are the alternatives? Are their questions we should ask before this one? Once again I'd be grateful for your thoughts. Roger -- ------------------------------------- Roger Hyam Technical Architect Taxonomic Databases Working Group ------------------------------------- http://www.tdwg.org roger(a)tdwg.org +44 1578 722782 -------------------------------------

5 9