September 2005 - tdwg-tag - lists.tdwg.org

Re: Different reasons for different GUIDs
by Nozomi Ytow 13 Sep '05

13 Sep '05

Donald and all, > However there is probably insufficient benefit for > us to gain from having a system where strong GUIDs are assigned to every > taxon concept that any provider wishes to reference. Exrepssion "every taxon concept" implies that we can establish identity of taxon concept. I don't think it is possible execpt potential taxon sensu Brendsohn (1995), i.e. combination of name string and publication. What we need for the benefit 3 is not GUID of 'each' taxcon cocnept but a mechanism to give an approxiamtion of a taxon concept/taxon concepts (I'm not convince how can I count taxon concepts) can be used as a granule. JMS -- Dr. Nozomi "James" Ytow Institute of Biological Sciences / Gene research center University of Tsukuba Tsukuba, Ibaraki 305-8572 Japan

1 0

Re: Different reasons for different GUIDs (Was: GUIDs, LSIDs, and metadata)
by Richard Pyle 13 Sep '05

13 Sep '05

Thanks, Donald. > There are different situations (use cases) in which we may require GUIDs. > The service characteristics of GUIDs may not always be the same. I agree, and this is true at two levels. At one level, the GUID needs may be different for different data domains (i.e., top-level object categories: Taxon Names, References, Specimens, Agents, etc.) At another level, the GUID needs for different use cases *within* a data domain may differ. > 3. Users can assume that data records relate to the same real-world object > if and only if they share the same GUID. > > I think that we have generally assumed that all our GUID uses will require > this additional function, but this is the part that will get very > expensive to implement. Why would it be so much more expensive to implement? > I believe that we should take a case-by-case decision on > whether the additional infrastructure is really required to support this > level of integration. I'm not sure I follow. Are you saying that it requires additional infrastructure to support the notion that "Users can assume that data records relate to the same real-world object if and only if they share the same GUID"? But under what circumstances could users ever assume that data records relate to the same real-world object when they do *not* share the same GUID? It would seem to me that it would require a human brain to reliably ascertain that GUID A refers to the same real-world object as GUID B. And once that determination is made, the "expensive" part is already done. At that point, the GUIDs should be (objectively) "synonymized", so that future users who reference either GUID do not have to invest the same expensive decision-making process involved with ascertaining their equivalency. Maybe I've completely misunderstood your point, but I guess what I'm struggling to understand is, when presented with two data objects with different GUIDs, under what circumstances could a user "assume" anything other than that they are not the same real-world object? > As an example, I can see excellent reasons for us at least to try for > benefit 3 for specimen records, and also probably in some way for taxon > names and publications (whatever actual form the identifiers for these may > take). However (I think, like Rod) I am not totally convinced > that we will > need to manage GUIDs offering benefit 3 for taxon concepts. O.K., now I think I understand where you are coming from (i.e., the "fuzzy" nature of taxonomic concept GUIDs, and what they really represent). As I have argued during the TCS/LC discussions, my feeling is that a Taxonomic Concept object should not receive a GUID that is disconnected from the [TaxonName]+[Publication] pair of GUIDs. In other words, I think that a Concept should be uniquely identified via a name-usage instance (specifically, the name-usage instance that defined the concept). But I may not be in the majority on this. > We should use > GUIDs (with benefits 1 and 2) for taxon concepts, and our taxon concepts > should reference names and publications which probably have the stronger > kind of GUIDs (with benefits 1-3). The coupling of a name GUID and a > publication GUID would then be a unique combination that would still allow > concepts to be compared. Exactly....so why assign a different GUID to the Concept? Or, perhaps put a different way, why assign a different GUID to the *name* (in the Taxonomer paradigm that names do not exist as stand-alone objects independent of usage instances)? > This is just an example, but indicates that we need to think hard > about the > use cases for each data object class and its associated GUIDs. Agreed! Aloha, Rich Richard L. Pyle, PhD Ichthyology, Bishop Museum 1525 Bernice St., Honolulu, HI 96817 Ph: (808)848-4115, Fax: (808)847-8252 email: deepreef(a)bishopmuseum.org http://hbs.bishopmuseum.org/staff/pylerichard.html

1 0

Different reasons for different GUIDs (Was: GUIDs, LSIDs, and metadata)
by Donald Hobern 13 Sep '05

13 Sep '05

Richard Pyle wrote: > One paradigm might have each major database create its own LSID: > > urn:lsid:catalogoffishes.org:SPNO:123456 > urn:lsid:gbif.org:ECAT:876543 > urn:lsid:itis.gov:TSN:567890 > > But then we're burdoned with the task of cross-mapping each of these, and > also preserving the legacy IDs into perpetuity after we've eventually > converged on a single taxon name GUID system. This indicates a general issue that we need to consider and which may make the whole problem a little simpler for us in some areas. There are different situations (use cases) in which we may require GUIDs. The service characteristics of GUIDs may not always be the same. It seems reasonable for us to think of a system which associates GUIDs with every data object shared by anyone using TDWG standards. These objects will represent a wide range of different types of record (specimen, collection, taxon name, taxon concept, synonymy assertion, publication, author, user, etc.). Use of genuinely _global_ identifiers in all of these cases will have the following benefits: 1. Users can reliably request associated data items simply by knowing the GUID assigned to them by the data provider. For example, I can view a specimen record and then request an associated taxon concept record identified within the specimen record by GUID. 2. It may simplify a data provider's own task in providing cross-references between data elements which are all under their own control. This is a compelling set of reasons for using GUIDs, and for their being resolvable. Such GUIDs should be relatively simple to manage and we could put together many different solutions that would satisfy these requirements. There is however a further requirement that may only apply in certain situations and for certain classes of data: support for _different_ data providers to re-use the _same_ GUID if they are sharing data for the same real-world object (collection, specimen, etc.). For example institutionA is the first body to reference a taxon concept published under the name Aus bus in publicationC. If institutionB subsequently wishes to reference Aus bus sensu publicationC, they would need to find and reuse the same idengtifier. The benefit from this would be as follows: 3. Users can assume that data records relate to the same real-world object if and only if they share the same GUID. I think that we have generally assumed that all our GUID uses will require this additional function, but this is the part that will get very expensive to implement. I believe that we should take a case-by-case decision on whether the additional infrastructure is really required to support this level of integration. As an example, I can see excellent reasons for us at least to try for benefit 3 for specimen records, and also probably in some way for taxon names and publications (whatever actual form the identifiers for these may take). However (I think, like Rod) I am not totally convinced that we will need to manage GUIDs offering benefit 3 for taxon concepts. We should use GUIDs (with benefits 1 and 2) for taxon concepts, and our taxon concepts should reference names and publications which probably have the stronger kind of GUIDs (with benefits 1-3). The coupling of a name GUID and a publication GUID would then be a unique combination that would still allow concepts to be compared. However there is probably insufficient benefit for us to gain from having a system where strong GUIDs are assigned to every taxon concept that any provider wishes to reference. Such a system would require every data provider to check a global registry to make sure that any particular taxon concept did not already have a GUID. This is just an example, but indicates that we need to think hard about the use cases for each data object class and its associated GUIDs. Donald --------------------------------------------------------------- Donald Hobern (dhobern(a)gbif.org) Programme Officer for Data Access and Database Interoperability Global Biodiversity Information Facility Secretariat Universitetsparken 15, DK-2100 Copenhagen, Denmark Tel: +45-35321483 Mobile: +45-28751483 Fax: +45-35321480 ---------------------------------------------------------------

1 0

Re: GUIDs, LSIDs, and metadata
by Roderic Page 12 Sep '05

12 Sep '05

On 12 Sep 2005, at 08:40, Peter Dawyndt wrote: > > It seems that the solutions worked into the StrainInfo.net portal have > many > common grounds with the problems encoutered with the integration of > taxonomic > names into a single coherent system. In this context I also recommend > the > Taxonomic Databases Working Group to take a look at the experimental > work done > by George Garrity of Bergey's Manual Trust to work bacterial taxonomic > names > into the DOI framework. After all, it seems to me that the DOI > framework > currently offers a far more extended framework of software solutions > and > organisational issues that outreach those of the LSIDs at present. An > essential > thing that is missing in the latter framework seems to be a > well-thought about > business plan to guarantee the long-term survival of the GUID system. > Also it > seems a bit like reinventing the wheel to me to overlook as system > that has > already gone through the 'proof-of-principle' stage. We already have a > morbid > growth of identifiers that are piling up our information systems, so > it would be > unwise to put our effort into the proliferation of network standards > that cover > the same domain. DOIs as presently implemented have limitations, especially the lack of any way of knowing what kind of output the DOI will yield. LSIDs specify that any LSID will resolve to RDF metadata, and also provides an explicit mechanism for downloading associated data. For example, the LSID urn:lsid:sid.zoology.gla.ac.uk:id:6 resolves to this metadata: <?xml version="1.0" encoding="utf-8"?><rdf:RDF xmlns:e="urn:lsid:sid.zoology.gla.ac.uk:predicates:" xmlns:a="http://purl.org/dc/elements/1.1/#" xmlns:d="http://www.w3.org/2000/01/rdf-schema#" xmlns:b="http://purl.org/dc/elements/1.1/" xmlns:c="urn:lsid:i3c.org:predicates:" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" > <rdf:Description rdf:about="urn:lsid:sid.zoology.gla.ac.uk:id:6"> <c:storedas rdf:resource="urn:lsid:sid.zoology.gla.ac.uk:id:6"/> <rdf:type rdf:resource="urn:lsid:i3c.org:types:content"/> <b:format>image/tiff</b:format> <d:label>6.TIF</d:label> <e:sem rdf:resource="urn:lsid:sid.zoology.gla.ac.uk:predicates:sem"/> <a:title>Image of Dennyus hirundinis</a:title> <a:date>1997-05-23</a:date> <a:creator>Vince Smith</a:creator> </rdf:Description> </rdf:RDF> The "storedas" tag tells us there is data associated with this LSID, the "format" tag tells us it is a TIFF image. If you resolve this LSID in Firefox (lsidres:urn:lsid:sid.zoology.gla.ac.uk:id:6), you'll be able to get the TIFF image (on a Mac OS X machine, or a PC with Quicktime installed, the image will load in the browser window, otherwise you will be prompted to save the image). The point of all of this is that I can write scripts or programs that can make use of LSIDs, for example, by aggregating them into a local knowledge base. To my mind, this is where the true power of LSIDs becomes apparent. I think there are at least three distinct issues to be considered: 1. Persistence, which is a social issue 2. Assigning and resolving GUIDs 3. What gets served when a GUID is resolved I think we can de-couple these, in the sense that we could use a handle system like that upon which DOIs are based, but adopt the RDF metadata approach of LSIDs. This thread has also raised the issue of mapping between multiple GUIDs. I think it is inevitable that we will have to deal with this, especially as there already exist major databases containing taxonomic information. For example, consider the task of mapping between mammalian names in DiGIR providers, and those used in GenBank (a relatively straightforward problem). In some lucky cases where we have specimen information in GenBank we can tie the two together that way, but for other names/sequences we aren't this lucky. If our databases are distributed, and run by organisations with different goals and agendas (I doubt biodiversity rates highly in NCBIs list of things to do), we will have to deal with this. > > Further reading: > > [1] P. Dawyndt, M. Vancanneyt, H. De Meyer & J. Swings (2005) Knowledge > Accumulation and Resolution of Data Inconsistencies during the > Integration of > Microbial Information Sources 17(8), 1111-1126. > http://doi.ieeecomputersociety.org/10.1109/TKDE.2005.131 > > [2] P. Dawyndt, M. Vancanneyt & J. Swings (2004). On the integration of > microbial information. WFCC Newsletter 38, 19-34. > http://wdcm.nig.ac.jp/wfcc/NEWSLETTER/newsletter38/a3.pdf > > [3] P. Dawyndt, B. De Baets, X. Zhou, J. Ma & J. Swings. > StrainInfo.net: Holding > a wealth of downstream information on microbial resources right in our > hands. > http://www.cpdr.ucl.ac.be/bioinf/papers/bioinf/bioinf_dawyndt.pdf > > Also check out the background document and discussion papers that came > out of > the specialist workshop on "Exploring and exploiting microbiological > commons: > contributions of bioinformatics and intellectual property rights in > sharing > biological information" at http://lmg.ugent.be/bioinf-ipr/ Interesting reading. Rod Professor Roderic D. M. Page Editor, Systematic Biology DEEB, IBLS Graham Kerr Building University of Glasgow Glasgow G12 8QP United Kingdom Phone: +44 141 330 4778 Fax: +44 141 330 2792 email: r.page(a)bio.gla.ac.uk web: http://taxonomy.zoology.gla.ac.uk/rod/rod.html reprints: http://taxonomy.zoology.gla.ac.uk/rod/pubs.html Subscribe to Systematic Biology through the Society of Systematic Biologists Website: http://systematicbiology.org Search for taxon names at http://darwin.zoology.gla.ac.uk/~rpage/portal/

1 0

Re: GUIDs, LSIDs, and metadata
by Peter Dawyndt 12 Sep '05

12 Sep '05

Although having a 1:1 mapping between identifiers and identifyable objects might seem the best option from a theoretical point of view, it often turn out to be utopia when it comes to practical situations ("I theory, there is no difference between theory and practice. In practice, there is." Chuck Reid). Therefore, avoiding the assignment of multiple identifiers to the same entity is often unavoidable, and in some cases even favourable for tracking and tracing purposes. It is up to the informatics community to build intelligent information systems that can cope with this sort of problems. Here's some story on how we are solving this kind of issues in the field of microbiology. As microbiologists work with living organisms that are transfered around the globe among research institutions and culture collections, different tags (called strain numbers) are assigned to a single isolate. From an information technological point of view it is thus favourable to link downstream information (literature references, experimental information (eg. sequences), administrative information) onto the specimen level, and not to taxonomic level, as the letter is vulnerable for change over time (due to different opinions, changing taxonomy and new identification technologies. As such, taxonomic status becomes decoupled from the downstream information with the specimen standing at the intermediate level. Coping with different strain numbers that are used to tag the same specimen, can be easily resolved by maintaining equivalence relation of the strain numbers in a central repository (just as you could do for keeping track of synonym taxonomic names). This is what is done in the Integrated Strain Database, where the equivalence relation is automatically managed by the application of accumulative learning principles (using calculation of the transitive closure for incremental placement of new strain number into equivalence classes). Currently, information is gathered from 42 microbial culture collections that cover all earths continents and range from small niche specific research collections to large general-purpose service collections. In addition, the information extracted from two lists of bacterial type strains is equally incorporated. This integration process has currently lumped over 600.000 strain numbers into some 250.000 equivalence classes that represent different strains of bacteria, archaea, filamentous fungi and yeasts. As we live in an imperfect world, special attention has been paid to error detection and correction within the equivalence classes due to irregularities in the data provided by the underlying information sources, through the design of novel intelligent tools that enable the automatic discovery of intrusions in the consistency of the integrated information. Just to give you an impression on the necessity of checking the information coming from different information sources: without profound quality control of the integrated information, at least 719 (11.89%) of the bacterial type strains would have been affected by illegitimate merges into single equivalence classes. While incrementally calculating the strain equivalence classes, new unique identifiers are assigned to strain numbers that were not previously encountered during the integration procedure. This helps to resolve some of the ambiguities that are a logical consequence of the local nature of the strain number assignment process and enables to set down context-dependant resolution of ambiguous strain numbers that often require some form of human-intervention. The latter is important to secure the tedious disambiguation procedure of existing cross-references for correct machine interpretation in the future. Moreover, it turns out that the information content of the Integrated Strain Database offers the perfect semantic context to guide the disambiguation process in a number of ways. To demonstrate the potential of this approach to fill the gap where there is no universally adopted system for assigning and recognizing persistent and unique identifiers for biological resources, we have set up a portal system called StrainInfo.net (www.straininfo.net), where we have consolidated the strain information captured within the Integrated Strain Database with relevant sequences and literature references assembled within public repositories. Not only does this offer a de-duplicated view on the downstream information that is available on the micro-organisms worldwide, but also allows for the execution of all sorts of dynamic queries that can automatically bridge over multiple web services that were physically separated before the integration process. The presented cross-reference model will however only show its full dynamic strength when the reverse references to the Integrated Strain Database are included in third party databases, thus establishing a true divide and conquer strategy for tracking related information within autonomously operating biological information sources. It seems that the solutions worked into the StrainInfo.net portal have many common grounds with the problems encoutered with the integration of taxonomic names into a single coherent system. In this context I also recommend the Taxonomic Databases Working Group to take a look at the experimental work done by George Garrity of Bergey's Manual Trust to work bacterial taxonomic names into the DOI framework. After all, it seems to me that the DOI framework currently offers a far more extended framework of software solutions and organisational issues that outreach those of the LSIDs at present. An essential thing that is missing in the latter framework seems to be a well-thought about business plan to guarantee the long-term survival of the GUID system. Also it seems a bit like reinventing the wheel to me to overlook as system that has already gone through the 'proof-of-principle' stage. We already have a morbid growth of identifiers that are piling up our information systems, so it would be unwise to put our effort into the proliferation of network standards that cover the same domain. Further reading: [1] P. Dawyndt, M. Vancanneyt, H. De Meyer & J. Swings (2005) Knowledge Accumulation and Resolution of Data Inconsistencies during the Integration of Microbial Information Sources 17(8), 1111-1126. http://doi.ieeecomputersociety.org/10.1109/TKDE.2005.131 [2] P. Dawyndt, M. Vancanneyt & J. Swings (2004). On the integration of microbial information. WFCC Newsletter 38, 19-34. http://wdcm.nig.ac.jp/wfcc/NEWSLETTER/newsletter38/a3.pdf [3] P. Dawyndt, B. De Baets, X. Zhou, J. Ma & J. Swings. StrainInfo.net: Holding a wealth of downstream information on microbial resources right in our hands. http://www.cpdr.ucl.ac.be/bioinf/papers/bioinf/bioinf_dawyndt.pdf Also check out the background document and discussion papers that came out of the specialist workshop on "Exploring and exploiting microbiological commons: contributions of bioinformatics and intellectual property rights in sharing biological information" at http://lmg.ugent.be/bioinf-ipr/ Cheers, Peter Dawyndt ------------------------------------------------------------------------------- Peter Dawyndt email: P e t e r . D a w y n d t @ U G e n t . b e phone: +32 (0)9 264 5132 fax: +32 (0)9 264 5092 contact addresses: Laboratory of Microbiology, Ghent University, K. L. Ledeganckstraat 35, B-9000 Ghent, Belgium. Department of Applied Mathematics, Biometrics and Process Control, Ghent University, Coupure links 653, B-9000 Gent, Belgium. -------------------------------------------------------------------------------

1 0

Another GUID system
by Roderic Page 12 Sep '05

12 Sep '05

Another GUID system that might be of interest is ARK (http://www.cdlib.org/inside/diglib/ark/). The site describing this approach has a nice discussion of the issues surrounding persistent identifiers, and various solutions such as DOIs and URNs (of which LSIDs are an example). I think this system should be looked at closely, before we limit discussion to DOIs or LSIDs. Regards Rod Professor Roderic D. M. Page Editor, Systematic Biology DEEB, IBLS Graham Kerr Building University of Glasgow Glasgow G12 8QP United Kingdom Phone: +44 141 330 4778 Fax: +44 141 330 2792 email: r.page(a)bio.gla.ac.uk web: http://taxonomy.zoology.gla.ac.uk/rod/rod.html reprints: http://taxonomy.zoology.gla.ac.uk/rod/pubs.html Subscribe to Systematic Biology through the Society of Systematic Biologists Website: http://systematicbiology.org Search for taxon names at http://darwin.zoology.gla.ac.uk/~rpage/portal/

1 0

Re: GUIDs, LSIDs, and metadata
by Kevin Richards 12 Sep '05

12 Sep '05

Good points. A few comments I have: I think LSIDs are assumed to solve all conflicts in the various datasets of taxonomic data. However they are JUST resolvable IDs, anything else is infrastructure surrounding the LSID mechanisms. An LSID refers to a specific set of bytes that resides on some computer somewhere. The assumption that an LSID will refer to, for eaxample, a global 'taxon concept' that all other taxon records should point to, is not correct. This relies on a system to be in place that provides the functionality for this global repository. Also I feel one argument AGAINST LSIDs is that the initial investment in infrastructure is large, ie the development and setting up of authorities, etc. So I think this would lean people away from LSIDs, bot towards them. The advantage with the LSID mechanism, I think, is that it is flexible enough to not rely on existing software and internet configuration. A GUID really needs to refer to a reasonably basic record, eg a name object rather than the entire taxon concept (although you could have a GUID for either). This allows these individual components to be referenced from other systems/datasets without having to refer to and accept the enitre concept. It is probably a good idea to map out which sort of taxonomic objects should get GUIDs and how they relate to other objects. Kevin Richards >>> deepreef(a)BISHOPMUSEUM.ORG 09/11/05 6:50 AM >>> Lots of good discuccion points on GUIDs -- thanks, Rod. I need to get two little people to two different soccer (football) games soon, so I have no time for an elaborate response. But I do want to comment on one point, which I have been thinking a great deal about lately: > 7. I think the first priority for assigning GUIDs is museum specimens. > For taxon names (if not concepts) this is trivial, given that most name > databases have their own, internally unique ids (but not all -- those > databases that use names as primary keys, or which don't expose integer > identifiers will need to rethink their design). I think it's critical that, whatever GUID system we establish for taxon names (and concepts), we do it in the context of the next several decades of informatic landscape; not just in the context of immediate needs or current political climate. As you said at the start of your message, GUIDs by themselves are trivial. So the only real difference between establishing a system that is intuitive for the current needs and a system that will serve longer-term future needs, is a little bit of careful forethought. Official taxon name registration already exists for one of the major Codes of Nomenclature (Bacterial), and within the next fortnight we will see a public announcement of a plan for registration in another of the major Codes. I predict that all Codes of nomenclature will implement mandatory registration for all new names by about 2010, and for all "available" names (i.e., since Linnaeus) within five to ten years thereafter. So the medium-term future landscape in this case will be one in which all names are issued a GUID through their respective Commission of Nomenclature. Further, it's not unreasonable to predict that sometime within the next few decades we will converge on a unified "BioCode" for all organism names, meaning that the longer-term landscape has a single set of taxon names. Wouldn't it be nice, after that time, if we didn't have to forever maintain legacy GUIDs? In other words, wouldn't it be nice if the established GUID system for all taxon names were the same *now*, at the outset, so it's a non-issue to combine them all as one set of GUIDs later on? I'm not entirely sold on LSIDs, but it does seem that a lot of smart and knowledgable people are leaning that way. My hesitation is mainly that one of the main reasons for leaning that way is that all sorts of software already exists for resolving them, so there is less overhead in initial implementation. As long as LSID meet long-term needs, that shouldn't be a problem. But 50 years from now, I'm not sure how wise it will seem that the universal GUID system adopted for biological data was influenced strongly by the available software of the time. Imagine being locked in now to a universal system that was designed based on software that was available in 1955! But, not being able to predict which GUID system will be the best in the context of 2055, we really have no choice but to go with something that makes a lot of sense now (which is justififable, in that it's also very important that the delicate transition from no universal GUIDs to widespread universal GUIDs will be best supported by keeping it as painless as possible in the context of that transition time). But I still suggest we do things in a way that maximally keeps our options open. For example, in the context of LSIDs, consider different paradigms for registring the fish name, Mygenus myspecies Hyam (Hi, Roger! :-) ) One paradigm might have each major database create its own LSID: urn:lsid:catalogoffishes.org:SPNO:123456 urn:lsid:gbif.org:ECAT:876543 urn:lsid:itis.gov:TSN:567890 But then we're burdoned with the task of cross-mapping each of these, and also preserving the legacy IDs into perpetuity after we've eventually converged on a single taxon name GUID system. I was going to illustrate several other paradigms, but soccer departure time approaches, so I'll cut to the chase. In the LSID paradigm, I would propose the following system: urn:lsid:bioregistry.org:[Data Domain]:[randomly generated 64-bit integer] The "bioregistry.org" part represents the decoupling of the GUID from the institution that initially created the GUID. It encompases all domains of biological data (taxon names, concepts, specimens, etc.). It could be "tdwg.org" or "gbif.org", but we're not sure those organizations will be around 50 or 100 years from now. I imagine that GBIF would create and manage the bioregistry.org domain for the near-term. The "Data Domain" represents a tag for the main domain of data (e.g. "Specimens", or "TaxonNames", or whatever the major information domains end up being). The randomly generated 64-bit integer would be unique across all data domains, so that it, by itself, is unique within bioregistry.org (no time now to explain the rationale for this...) Gotta run....more later. Aloha, Rich ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ WARNING: This email and any attachments may be confidential and/or privileged. They are intended for the addressee only and are not to be read, used, copied or disseminated by anyone receiving them in error. If you are not the intended recipient, please notify the sender by return email and delete this message and any attachments. The views expressed in this email are those of the sender and do not necessarily reflect the official views of Landcare Research. Landcare Research http://www.landcareresearch.co.nz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

1 0

Re: GUIDs, LSIDs, and metadata
by Richard Pyle 11 Sep '05

11 Sep '05

> This thread has also raised the issue of mapping between multiple > GUIDs. I think it is inevitable that we will have to deal with this, I absolutely agree. But at the same time, I think we should try (at least) to minimize duplicate GUID issuance to the same object (and we should certainly not encourage it!) > especially as there already exist major databases containing taxonomic > information. For example, consider the task of mapping between > mammalian names in DiGIR providers, and those used in GenBank (a > relatively straightforward problem). I think it would be a mistake to try to map DiGIR specimen/observation instances directly to GenBank sequences via taxon names (although I think GenBank sequences should be mapped directly to the specimen record from which it was drawn, but that's independent of the taxonomy). Putting aside the question of whether gene sequence blocks should fall within the same data domain as specimen/observation objects, in both cases the taxon name is a secondary attribute. Instead, owners of each record should map their objects (specimen or sequence) to the same universal GUID for the taxon name (or, preferably, to the same taxon concept GUID) to which the specimen/observation/sequence instance has been assigned. That way, when someone queries on the name (or concept), the relevant DiGIR and GenBank objects show up in the results because they are mapped via a common taxon GUID. Consider the alternative where the DiGIR provider created its own taxon GUID, separate from the taxon GUID assigned for the GenBank sequence. We'd still be left with the task of mapping those two separate GUIDs as representing the same taxon object (be it a name or a concept). > In some lucky cases where we have > specimen information in GenBank we can tie the two together that way, Agreed! But that's a completely separate issue from how either instance is mapped to a taxon GUID. Maybe not completely separate. I don't think gene sequences should be considered in the same data domain as speciemns. They fit better with morphological characters. In the ideal world (and admittedly, this may be out of reach in the immediate future). Neither sequences nor morphological characters should link directely to taxon objects (names or concepts), but rather inherit taxonomic attributes from a specimen object to which they are attached (regardless of whether the specimen was or was not vouchered in a museum). Like I said, I'm probably reaching too far on this one. > but for other names/sequences we aren't this lucky. If our databases > are distributed, and run by organisations with different goals and > agendas (I doubt biodiversity rates highly in NCBIs list of things to > do), we will have to deal with this. Agreed. And I think the cleanest way to deal with it is for the "public" data domains (literature citations and taxon names & concepts) to be established via a single mechanism for assigning GUIDs (shooting as best we can for 1:1 GUID:object instance), and then all of the "private" data domains (specimens, sequences, characters, microbial cultures, etc.) be managed by their respective data owners, and the onus would be on them to map their taxonomic & literature links to the common universal GUID system. Aloha, Rich P.S. I promised I would shutup, and I apologize for breaking that promise. Richard L. Pyle, PhD Database Coordinator for Natural Sciences Department of Natural Sciences, Bishop Museum 1525 Bernice St., Honolulu, HI 96817 Ph: (808)848-4115, Fax: (808)847-8252 email: deepreef(a)bishopmuseum.org http://hbs.bishopmuseum.org/staff/pylerichard.html

1 0

Re: GUIDs, LSIDs, and metadata
by Roderic Page 11 Sep '05

11 Sep '05

On 11 Sep 2005, at 20:55, Kevin Richards wrote: > I think LSIDs are assumed to solve all conflicts in the various > datasets of taxonomic data. However they are JUST resolvable IDs, > anything else is infrastructure surrounding the LSID mechanisms. An > LSID refers to a specific set of bytes that resides on some computer > somewhere. The assumption that an LSID will refer to, for eaxample, a > global 'taxon concept' that all other taxon records should point to, > is not correct. This relies on a system to be in place that provides > the functionality for this global repository. > I hope people don't make this assumption, because it's obviously erroneous. LSIDs just provide a mechanism to assign GUIDs to metadata and data. Whether there will be a global taxon concept is a completely different question. I also doubt that deferring to some central authority is the best way forward. I suspect the very nature and scale of the task makes a distributed approach inevitable. > Also I feel one argument AGAINST LSIDs is that the initial investment > in infrastructure is large, ie the development and setting up of > authorities, etc. So I think this would lean people away from LSIDs, > bot towards them. The advantage with the LSID mechanism, I think, is > that it is flexible enough to not rely on existing software and > internet configuration. > Actually it's pretty easy to do this. Basically it can be down with a bunch of Perl scripts, and some fiddling with the DNS. If you can program CGI scripts, you can implement LSIDs (like I said, if an amateur like me can do it, it can't be that hard). Any GUID is going to need some mechanism for associating the GUID with data, and a mechanism to ensure uniqueness and persistence of the GUID. However, they do represent more work than, say, simply using URLs. However, I suggest that any solution is basically going to require (a) some way of resolving a GUID, and (b) some way to return metadata, data, or both, about a GUID. LSIDs can do this for us now. > A GUID really needs to refer to a reasonably basic record, eg a name > object rather than the entire taxon concept (although you could have a > GUID for either). This allows these individual components to be > referenced from other systems/datasets without having to refer to and > accept the enitre concept. It is probably a good idea to map out > which sort of taxonomic objects should get GUIDs and how they relate > to other objects. > One could have GUIDs for names and concepts. I think ultimately anything that is worth making available to other people will/should get GUIDs. This is, after all, why the web is so powerful -- bits of data have GUIDs (albeit often rather fragile) in the form of URLs, and people make use of them by linking (just look at blogs and RSS feeds as the latest illustration of this power). Maybe there's a confusion here between globally unique identifiers (a way of uniquely identifying a bit of data), and a global authority (specifying a particular view)? Lastly, while GBIF and/or the commissions for the various codes of nomenclature may feel they are the obvious authorities for serving information on taxonomic names, it's not obvious to me that they will, in fact, be so. Are we really to expect that the commissions will be issuing GUIDs for all names within 10-15 years? Are we expected to wait for them, when technically there's no reason why they couldn't start doing this tomorrow? The notion that we should wait for these bodies to get their act together, and that we should defer to them strikes me as a recipe for disaster (or at least inertia). There are various efforts already underway out there, and perhaps we need a little healthy competition and exploration of alternatives. I suspect this area will be driven by users and data providers addressing their actual needs, rather than from "on high". I take Richard's point that it would be nice to get this right, but not at the cost of not actually doing something. And regarding legacy GUIDs, in the case of LSIDs this can be handled fairly easily via the DNS. It's rather like the case when company a.com buys company b.com, the DNS record for b.com is changed to map to a.com I think we also need to be careful about the idea of a central registry of GUIDs if this means that a single body will be responsible for issuing them. There are a range of alternatives, such as the DOI model. DOIs have two parts, one generated centrally, the other by the data provider. There is a central repository of metadata associated with DOIs (http://www.crossref.org), rather like GBIF has a local copy of data provided by DiGIR server. However, local providers are responsible for providing the content that corresponds to a DOI, and for constructing the second part of the DOI. In a sense this is pretty much what my Taxonomic Search Engine does -- it generates LSIDs for the databases that it queries, but retrieves the metadata on the fly from the data providers. This note is starting to lack whatever coherence it might have had at the start. Perhaps it's time to have some real examples to play with... Regards Rod > Kevin Richards > >>>> deepreef(a)BISHOPMUSEUM.ORG 09/11/05 6:50 AM >>> > Lots of good discuccion points on GUIDs -- thanks, Rod. I need to get > two > little people to two different soccer (football) games soon, so I have > no > time for an elaborate response. But I do want to comment on one point, > which I have been thinking a great deal about lately: > >> 7. I think the first priority for assigning GUIDs is museum specimens. >> For taxon names (if not concepts) this is trivial, given that most >> name >> databases have their own, internally unique ids (but not all -- those >> databases that use names as primary keys, or which don't expose >> integer >> identifiers will need to rethink their design). > > I think it's critical that, whatever GUID system we establish for taxon > names (and concepts), we do it in the context of the next several > decades of > informatic landscape; not just in the context of immediate needs or > current > political climate. > > As you said at the start of your message, GUIDs by themselves are > trivial. > So the only real difference between establishing a system that is > intuitive > for the current needs and a system that will serve longer-term future > needs, > is a little bit of careful forethought. > > Official taxon name registration already exists for one of the major > Codes > of Nomenclature (Bacterial), and within the next fortnight we will see > a > public announcement of a plan for registration in another of the major > Codes. I predict that all Codes of nomenclature will implement > mandatory > registration for all new names by about 2010, and for all "available" > names > (i.e., since Linnaeus) within five to ten years thereafter. So the > medium-term future landscape in this case will be one in which all > names are > issued a GUID through their respective Commission of Nomenclature. > > Further, it's not unreasonable to predict that sometime within the > next few > decades we will converge on a unified "BioCode" for all organism names, > meaning that the longer-term landscape has a single set of taxon names. > Wouldn't it be nice, after that time, if we didn't have to forever > maintain > legacy GUIDs? In other words, wouldn't it be nice if the established > GUID > system for all taxon names were the same *now*, at the outset, so it's > a > non-issue to combine them all as one set of GUIDs later on? > > I'm not entirely sold on LSIDs, but it does seem that a lot of smart > and > knowledgable people are leaning that way. My hesitation is mainly > that one > of the main reasons for leaning that way is that all sorts of software > already exists for resolving them, so there is less overhead in initial > implementation. As long as LSID meet long-term needs, that shouldn't > be a > problem. But 50 years from now, I'm not sure how wise it will seem > that the > universal GUID system adopted for biological data was influenced > strongly by > the available software of the time. Imagine being locked in now to a > universal system that was designed based on software that was > available in > 1955! > > But, not being able to predict which GUID system will be the best in > the > context of 2055, we really have no choice but to go with something that > makes a lot of sense now (which is justififable, in that it's also very > important that the delicate transition from no universal GUIDs to > widespread > universal GUIDs will be best supported by keeping it as painless as > possible > in the context of that transition time). > > But I still suggest we do things in a way that maximally keeps our > options > open. For example, in the context of LSIDs, consider different > paradigms > for registring the fish name, Mygenus myspecies Hyam (Hi, Roger! :-) ) > > One paradigm might have each major database create its own LSID: > > urn:lsid:catalogoffishes.org:SPNO:123456 > urn:lsid:gbif.org:ECAT:876543 > urn:lsid:itis.gov:TSN:567890 > > But then we're burdoned with the task of cross-mapping each of these, > and > also preserving the legacy IDs into perpetuity after we've eventually > converged on a single taxon name GUID system. > > I was going to illustrate several other paradigms, but soccer > departure time > approaches, so I'll cut to the chase. In the LSID paradigm, I would > propose > the following system: > > urn:lsid:bioregistry.org:[Data Domain]:[randomly generated 64-bit > integer] > > The "bioregistry.org" part represents the decoupling of the GUID from > the > institution that initially created the GUID. It encompases all > domains of > biological data (taxon names, concepts, specimens, etc.). It could be > "tdwg.org" or "gbif.org", but we're not sure those organizations will > be > around 50 or 100 years from now. I imagine that GBIF would create and > manage the bioregistry.org domain for the near-term. > > The "Data Domain" represents a tag for the main domain of data (e.g. > "Specimens", or "TaxonNames", or whatever the major information > domains end > up being). > > The randomly generated 64-bit integer would be unique across all data > domains, so that it, by itself, is unique within bioregistry.org (no > time > now to explain the rationale for this...) > > Gotta run....more later. > > Aloha, > Rich > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > +++++ > WARNING: This email and any attachments may be confidential and/or > privileged. They are intended for the addressee only and are not to be > read, > used, copied or disseminated by anyone receiving them in error. If > you are > not the intended recipient, please notify the sender by return email > and > delete this message and any attachments. > > The views expressed in this email are those of the sender and do not > necessarily reflect the official views of Landcare Research. > > Landcare Research > http://www.landcareresearch.co.nz > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > +++++ > > Professor Roderic D. M. Page Editor, Systematic Biology DEEB, IBLS Graham Kerr Building University of Glasgow Glasgow G12 8QP United Kingdom Phone: +44 141 330 4778 Fax: +44 141 330 2792 email: r.page(a)bio.gla.ac.uk web: http://taxonomy.zoology.gla.ac.uk/rod/rod.html reprints: http://taxonomy.zoology.gla.ac.uk/rod/pubs.html Subscribe to Systematic Biology through the Society of Systematic Biologists Website: http://systematicbiology.org Search for taxon names at http://darwin.zoology.gla.ac.uk/~rpage/portal/

1 0

Re: GUIDs, LSIDs, and metadata
by Richard Pyle 11 Sep '05

11 Sep '05

Kevin Richards wrote: > > somewhere. The assumption that an LSID will refer to, for eaxample, a > > global 'taxon concept' that all other taxon records should point to, > > is not correct. This relies on a system to be in place that provides > > the functionality for this global repository. Rod Page replied: > I hope people don't make this assumption, because it's obviously > erroneous. LSIDs just provide a mechanism to assign GUIDs to metadata > and data. Whether there will be a global taxon concept is a completely > different question. I also doubt that deferring to some central > authority is the best way forward. I suspect the very nature and scale > of the task makes a distributed approach inevitable. I'm not sure I understand, in the context of this dicussion, what the phrase "global 'taxon concept' that all other taxon records should point to" is intended to mean. If it means a single global taxonomy with a single "officially sanctioned" taxon concept circumscription for each taxon name that everyone must conform to, then certainly I agree that this should not (and cannot) be implemented. However, I DO see value in designating a 'global' GUID to each defined Taxonomic Concept (e.g., "Mygenus myspecies Hyam 2004 SEC Pyle 2005"), and having all other taxon records in databases around the world point to this same GUID whenever they specifically want to reference that particular defined concept ("Mygenus myspecies Hyam 2004 SEC Pyle 2005"). Whether that is a dedicated GUID, or a union of a Taxon Name GUID ("Mygenus myspecies Hyam 2004") and a Literature GUID ("Pyle 2005"), is an issue that needs to be discussed. The problem of relying on a (central) system to be in place can be dramatically mitigated if that system is mirrored with robust synchronization protocols around the world. The effort and maintainence should definitely be distributed among the taxonomic experts of the world. > Lastly, while GBIF and/or the commissions for the various codes of > nomenclature may feel they are the obvious authorities for serving > information on taxonomic names, it's not obvious to me that they will, > in fact, be so. Are we really to expect that the commissions will be > issuing GUIDs for all names within 10-15 years? Are we expected to wait > for them, when technically there's no reason why they couldn't start > doing this tomorrow? The reason (I think) we don't want to start issuing them tomorrow is that if there is no effort in place to make sure the same object instance isn't receiving multiple GUIDs from multiple issuers, then there's no real point in assigning the GUIDs in the first place. Each major database has its own internal LUIDs already. The work is in cross-mapping all of these LUIDs to each other, so we can more esily exchange information. If each database holder were to assign LSIDs to all of their records, what problem have we solved? OK, so the IDs attached to each record are guaranteed to be globally unique, and in some way embed resolving metadata (in the cae of LSIDs), and all of the database holders at least conform to a common system of IDs. But these gains are trivial compared to the monumental task of cross-linking all of the object instances that exist redudantly in dozens of databases around the world. My feeling is that the "brass ring" of one GUID per object instance (i.e, a single common "flag pole" around which all data holders can rally, and cross-link their own LUIDs to) is very much within reach, and is what will be needed in the long run anyway. No single organization can be relied upon for safekeeping of "the" master database into perpetuity. That's why "the" database needs to be mirrored all over the world. All that's needed is a robust synchronization protocol. The role of the Code Commissions, and GBIF, and TDWG would be to define the protocols and standards, and establish the initial implementations. They should not, in my opinion, be put into a position where thy must be relied upon over the next decades/centuries in order to facilitate the perpetual exchange of data. > The notion that we should wait for these bodies to get their act > together, and that we should defer to them strikes me as a recipe for > disaster (or at least inertia). There are various efforts already > underway out there, and perhaps we need a little healthy competition > and exploration of alternatives. I suspect this area will be driven by > users and data providers addressing their actual needs, rather than > from "on high". I take Richard's point that it would be nice to get > this right, but not at the cost of not actually doing something. And > regarding legacy GUIDs, in the case of LSIDs this can be handled fairly > easily via the DNS. It's rather like the case when company a.com buys > company b.com, the DNS record for b.com is changed to map to a.com I also see your point about the inertia problem. But I've always thought that the paradigm of independent solutions to this issue by competitive and disconnected efforts, as has been ongoing for the past few decades, is exactly the sort of chaos that has lead to our current data exchange problems that we are now trying to solve. My feeling is that we are now ready move past that phase and into the next phase. GBIF now has nearly $1.5 million specificlly to solve this problem, and at least one of the historically "inertia-ladden" Commissions is about to take a dramatic step forward. It feels to me like we're rapidly approaching critical mass, and personally I'd like to see how far we can push it forward, and capitalize on the new paradigm by simultaneously solving as many problems as we can all at once. > I think we also need to be careful about the idea of a central registry > of GUIDs if this means that a single body will be responsible for > issuing them. There are a range of alternatives, such as the DOI model. > DOIs have two parts, one generated centrally, the other by the data > provider. There is a central repository of metadata associated with > DOIs (http://www.crossref.org), rather like GBIF has a local copy of > data provided by DiGIR server. However, local providers are responsible > for providing the content that corresponds to a DOI, and for > constructing the second part of the DOI. In a sense this is pretty much > what my Taxonomic Search Engine does -- it generates LSIDs for the > databases that it queries, but retrieves the metadata on the fly from > the data providers. My reasons for looking at 64-bit integers is that there are ~10^19 of them to go around. I can see them being issued to any institution who wants them at, say, a billion numbers at a time (that allows for ~10 billion such blocks of 1 billion numbers/block). Each insititution/individual would then assign them to data objects however they want, whenever they want. If they've assigned the numbers to objects in conformance with TDWG standards (yet to be developed), then the associated TDWG-compliant data/metadata for each number can be uploaded to any one of the mirror servers, at which time the link between the number and the data/metadata gets automatically propagated to all of the mirror servers. The point is, the only time when a single entity needs to be relied upon is the initial issuance of blocks of numbers. And even this could be distributed (e.g., by pre-distributing blocks of ~10^17 numbers to each of ~100 different issuers). My reason for thinking in terms of simple integers is that they allow flexibility for embedding within different GUID schemes, if the TDWG standard for the GUID "package" needed to change in the future (i.e., the numbers could remain the same, and the resolving metadata packaging can change). Maybe I'm off my rocker here (very possible). But it seems so simple and straightforward, and seems (to me, anyway) to leave options open for future GUID packaging schemes. > This note is starting to lack whatever coherence it might have had at > the start. Perhaps it's time to have some real examples to play with... Ditto!! My apoligies to all for the rambling (it's a slow Sunday afternoon here). I'll shutup now. Aloha, Rich

1 0