- tdwg-content - lists.tdwg.org

Re: BioGUIDs and the Internet Analogy
by Chuck Miller 28 Sep '04

28 Sep '04

I think we just need to be clear that there are on the one hand the BioGUIDs themselves and how they are created/assigned and on the other hand is the method to find them later by someone else. On the Internet there are IP addresses and how they are assigned (albeit potentially dynamically) and the method for someone else to find them. Rarely do you get Internet references given as an IP address (http://157.140.2.10) (certainly one reason is that they can be dynamic) but rather as URLs (http://www.tdwg.org) that are then resolved through the domain resolution process. This approach enables the two things (number and name) to be managed separately, yet linked together. Don't we need the same two mechanisms for BioGUIDs? Does the LSID spec address both parts of this? I guess I had better get over there and read the whole thing. Regarding: >>IP addresses are in no way persistent. >>IP addresses are not globally unique. [But they are unique on the Internet] Making analogies is tough because the details of the example can get in the way, as in this case. IP addresses have several attributes. The point is not whether IP addresses are analogous to BioGUIDs on the attributes of static/changeable or universality. The point is that the IP is analagous because it is a cryptic non-intelligible and unique number used to locate another network device by a machine that starts out with no idea where it is. That's the analogy - that IP and BioGUIDs are unique, nonintelligible (ie. semantically opaque) strings of characters/numbers. The issue is how in the world to you find the BioGUID you are trying to get to? Do you just use the nonintelligible string? Or do we want somewhat intelligible names as pointers to the nonintelligible? Regarding: >>The number of organizations to be issued Bio GUIDs surely is several orders of magnitude less than those to be issued IPv6 addresses. True. But, there are potentially billions of BioGUIDs to be created, which could exceed the number of IPv6 addresses. Some institutions would be the equivalent of a large Internet Registry managing millions of BioGUIDs. This seems like a big issue to me. Would GBIF/TDWG or whomever issue millions/billions of BioGUIDs directly to institution records and not decentralize by assigning blocks of BioGUIDs to institutions? Also, when there are billions of something, efficiency becomes an issue. IP addresses follow a binary number approach that leads to efficient processing. Is there an efficiency issue lurking ahead for the BioGUID? Regarding: >>...name resolution protocols matter, but resolution implementations don't... This is true when you have working resolution implementations as the Internet does. But does LSID have a working resolution implementation now? If not, creating a new one that works is not that simple I think. That would be a good reason for piggy-backing on the existing and working Internet DNS/domain name resolution system as I've heard mentioned. Chuck Miller CIO Missouri Botanical Garden -----Original Message----- From: Bob Morris [mailto:ram@CS.UMB.EDU] Sent: Monday, September 27, 2004 10:34 PM To: TDWG-SDD(a)LISTSERV.NHM.KU.EDU Subject: Re: BioGUIDs and the Internet Analogy [For me, the bottom line---which however I nowhere state below---is: There is /so much/ existing free infrastructure source code---e.g. http://www-124.ibm.com/developerworks/oss/lsid/--- and (apparently) funding, and (manifestly) professionally designed specifications for LSID concerns that I am horrified at the prospect of adopting anything else if LSID comes even close to being what the community needs. Or, let's forget about LSID and instead of deploying what satisfies 98% of the needs in six months, we could roll our own and deploy what satisfies 80% of the needs in a few years... To my mind, the only question is this: is LSID good enough? So: first make the requirements. Then examine existing solutions. Arguing by analogy can lead to the "hammer" solution, usually attributed to Mark Twain: When all you own is a hammer, every problem begins to look like a nail. (Can someone give me an actual /reference/ to that quote????] ------------------------- The idea of modeling taxonomic uuids on the internet has been around for about 10 years, and has been written about explicitly for at least 3-4 by Hannu Saarenma and probably others. See his article in http://reports.eea.eu.int/technical_report_2001_70/en/Technical%20Report%207 0%20web.pdf The Tree of Life http://www.tolweb.org is a defacto such model, though without name authorities or persistence. The design goals of TCP/IP and DNS, and their implementation, intersect the requirements of Bio UUIDs only in a very small set, in fact, deep down perhaps not at all. These protocols and the associated address syntax were designed primarily for /routing/, not in any way designed to guarantee that a datum twice received has any connection between the two occurrences. IP addresses are in no way persistent. IP addresses are not globally unique, albeit in several small and varied ways: - The "private address blocks" 10.X.Y.Z, 172.16.0-0-172.31.255.255, and 192.168.X.Y may be assigned to any machine your and my network administrator care to, as long as neither is served on "the Internet", nor have an duplication on "an internet". -every machine implementing IP, besides whatever other addresses it may have, is always known to itself as 127.0.0.1; - the addresses in some IP address ranges are reserved to designate /many/ machines (multicast IP)). In general the design of the IP "nomenclature"---i.e. the IP addresses---is designed to solve routing problems, not identification problems. IP address syntax is intentionally not "semantically opaque", contrary to a requirement (well, a "should be") of LSIDs and well it should be. See Section 8 of http://www.omg.org/cgi-bin/doc?dtc/04-05-01 [If you don't see why IP addressing is not semantically opaque, try this exercise: With a single machine instruction (on most machines) how can you determine whether or not an IP address is in the Class B private address space(the 172 stuff above)? Sysadmin's, C programmers, and their families and employees are not eligible for this competition] In turn, DNS protocols are designed only to aid discovery of IP addresses. DNS addresses are also far from persistent (in fact, DNS records held anywhere have a "Time To Live" field which must be counted down until expiration, at which time the holder must acquire a new instance of the assignment of an IP address to a domain name.). [More below, interspersed] Richard Pyle wrote: >>Perhaps it would be useful to look at the issues being discussed about >>a bio identifier/locator/GUID in comparison to the same things that >>are needed for Internet communications. > > > I've long thought that parts of the DNS system would be extremely > useful to emulate in some aspects of bioinformatics data management > (particularly taxonomic names; see below). > > >>IP addresses have to be unique world-wide to make the Internet work. >>The Internet Corporation for Assigned Names and Numbers (ICANN- > > www.icann.org) > >>provides that uniqueness by assigning all the IP numbers in unique >>blocks > > or > >>ranges of numbers to "Internet Registries". > > > ...exactly the way that I envision an organization like GBIF would be > charged with the task of issuing UIDs for certain biological objects. > > >>There are Regional, National and Local Internet Registries that >>subdivide > > and > >>"license" IP addresses to ISPs, who in turn license IP addresses to > > organizations. Ah, this is accurate mainly for IPv6, which is much less chaotic than IPv4 (most of the current Internet) and in turn from the nearly formless void that was IPv2. But again, ultimate "licensees" do not get persistent IP addresses. In the US, virtually all dialup users get a different IP every time they connect, and most home broadband users only accidently keep their IP addresses, and only if they don't disconnect very long from the network. It's well worth comparing the design goals of IPv6 as articulated in http://www.apnic.net/docs/policy/ipv6-address-policy.html with those of LSID as articulated in Section 8 of the Draft Final Specification http://www.omg.org/cgi-bin/doc?dtc/04-05-01 > > There could be a useful analog for this in bioinformatics > (particularly in terms of individual institutions serving as regional > registries for specimen UIDs, or IC_N Commissions serving as > "regional" registries for taxon name UID assignment) -- but there > doesn't necessarily have to be. > In fact, if the UUIDs are meant to be semantically opaque it matters not one whit who or how these matters are settled. Exceptions to that are social, not technical. ("If you don't let me decide X, I am not going to use your scheme". "OK, then you won't participate in its benefits. That's fine with me") If you want to see another example of lack of semantic opacity, read the ISBN standard ISO/TC 46/SC 9 N 326. Part of certain ISBNs can help you determine an allegedly common publishing-germaine attribute of the US, Zimbabwe, Puerto Rico, Ireland, Swaziland, part of Canada, and a few other "regions". > >>So, there is a heirarchy of how the "unique identifiers" are managed. > > There is > >>in fact a central authority, but it delegates to decentralized > > authorities. But this is mainly to distribute costs and speed issuance. It has nothing to do with the naming scheme. The number of organizations to be issued Bio GUIDs surely is several orders of magnitude less than those to be issued IBv6 addresses. So I doubt any IPv6 issuance mechanisms are instructive, at least in their purpose (and hence, if well implemented, in their implementation). > > To emulate this in bioinformatics, the "hierarchy" would be achieved > simply by allowing block-assignment of UIDs to various players -- but > the important point here is that only *one* organization ensures > uniqueness (in the case of Internet, of ISPs). The data to which > those UIDs apply would be, for the most part, the responsibility of > the UID recipient, not the UID issuer (in my world view). Thus: > centralized issuance; delegated application. > > >>Is there an analogy for BioGUIDs to have a central body who divvies >>out > > the > >>unique numbers (like IP addresses) to decentralized bodies or large > > organizations? The International ISBN Organization http://www.isbn-international.org/ is roughly the IPv6 model. > > GBIF seems to me to be the principle contender. I enthusiastically agree. Also the /principal/ contender. [Sorry, couldn't resist. My fingers slip on that one sometimes too.] > > >>Since IP addresses are hard to memorize (and so too would be a >>BioGUID), > > "domain names" > >>are used. Starting with a domain name, you can first find the name >>and/or > > IP address > >>of a device, called the Domain Name Server, that can locate the IP >>address > > of other > >>computers. This is a form of indirect addressing. ICANN also manages >>the > > top-level > >>namespace for the Internet. They decide what the valid domain >>"extensions" > > are (like > >>.com, .uk) so that everybody, everywhere knows where to look them up. > > Then, the domain > >>name extensions are separated among the Regional, National, and Local > > Interent Registries > >>around the world. There is a scheme for where to find the IP >>addresses > > for every domain > >>extension (e.g. .com is on the ARIN registry, .com.uk is on the ). Not exactly. There is one scheme in case your application can't resolve it in a more nearly "local" facility. There are /lots/ of ways to find an IP address from a domain name. All those which comply fully with the DNS protocol, however, can make available two pieces of metadata: the TTL of the record it is offering, and the IP address of a machine at which you can find an authoritative record of the assignment of the dns name to the IP address. This protocol /might/, but you hope on performance grounds usually /doesn't/, lead you up as far as the root servers, and the "one scheme to bind them all". If there is any lesson here at all, it is that name resolution protocols matter, but resolution implementations don't. Yet another attribute on which, DNS/IP and LSID are not distinguishable. >>Then there is a layer of Domain Registrars who have been accredited by > > ICANN to assign > >>domain names for the domain extensions - e.g. tdwg.org. >>The domain name registrars are told by the owner of the domain where >>to > > find their particular > >>Domain Name Servers which may be many to enable redundancy - Primary, > > Secondary, Tertiary, > >>etc. Not quite. Normally, the registrant tells the registrar who has agreed to be the servers. The other case sometimes happens with "retailers" who are selling individuals domain names and ISP services at the same time. These redundant Domain Name Servers synchronize with each other at > > particular times > >>of day and may be located all around the world. More often, only when the TTLs expire, there being no motivation to do otherwise. >They are the main > > "switchboard" for a > >>particular organizations computer names and associated IP addresses. >>Then the individual organization can create multiple computers for the > > domain name - e.g. > >>www.tdwg.org - and add them to the Domain Name Server listing. There >>can > > be many computers > >>for a domain, for instance: info.tdwg.org, www2.tdwg.org, >>myname.tdwg.org. > > Each of these > >>can be a different computer with a different IP address. The >>redundant > > Domain Name Servers > >>all contain the list of all these names and what IP addresses they >>are. Not usually. The primary and secondary name servers would normally only cache tdwg.org permanently. They might /acquire/ a record for www.tdwg.org in response to a request, but they would not in general renew it after it expired and maybe not even keep it that long. To do so would be hideously unscalable. If I put 10,000 machines in my domain, my primary and secondary would be mighty unhappy if they had to keep them all cached. > > > This is analogous in many ways to how I would envision a global > taxonomic name service. UIDs are assigned by a centralized body > (e.g., GBIF; or by the IC_N Commissions) to individual names. > Analogous to multiple redundant Domain Name Servers (DNS) would be > Taxon Name Servers (TNS). Rather than administered by one > organization (e.g., GBIF, ITIS, Species 2000, uBio, > etc.) these TSNs would be replicated on dozens or hundreds of servers all > over the world, and maintained as synchronized within some reasonable time > unit. Changes to any one replicate would be automatically propagated to all > replicates (either chaotically, or more strictly through one or a few > defined "hubs"). Instead of Domain names as surrogates for IP addresses, > there would be fully qualified "Basionyms" (e.g., > "OriginalGenusName.OriginalSpeciesName.OriginalAuthor.DescriptionYear.Page.O > therOriginalCitationDetailsAsNeeded") representations of the > less-human-friendly GUIDs (analogues to IP addresses). Ideally, this system > wouldn't be limited to just taxonomic names, but extended to all taxonomic > concepts, so that the "Domain Name" analogue would be extended to something > like: > > "OriginalGenusName.OriginalSpeciesName.OriginalAuthor.DescriptionYear. > Page.O > therOriginalCitationDetailsAsNeeded_AppliedGenusName.AppliedSpeciesSpelling. > ConceptAuthor.ConceptYear.Page.OtherConceptCitationDetailsAsNeeded" > > >>The players in the Internet networking fabric all now play by these > > layered rules. > >>They all know them and follow them in order to keep the Internet >>running. > > This > >>stuff happens out of sight to everyone but the networking people and >>we > > all take it > >>for granted and assume it is simple. But, it's invisible not because >>it's > > simple, > >>but rather because it's disciplined. Agreed. Yet one more attribute where IP/DNS and LSID are not distinguishable. > > > Excellent synopsis, and (in my opinion), and excellent model to follow > for at least taxonomic names/concepts data. Perhaps also for specimen > data (but seems less intuitive for that.) This comes back to my > earlier question about whether it is vital that all bioinformatics > GUIDs be of the same scheme; or whether different schemes might be > optimal for different classes of objects. > > Aloha, > Rich > > Richard L. Pyle, PhD > Natural Sciences Database Coordinator, Bishop Museum > 1525 Bernice St., Honolulu, HI 96817 > Ph: (808)848-4115, Fax: (808)847-8252 > email: deepreef(a)bishopmuseum.org > http://www.bishopmuseum.org/bishop/HBS/pylerichard.html

1 0

Re: BioGUIDs and the Internet Analogy
by Richard Pyle 28 Sep '04

28 Sep '04

> [For me, the bottom line---which however I nowhere state below---is: > There is /so much/ existing free infrastructure source code---e.g. > http://www-124.ibm.com/developerworks/oss/lsid/--- and (apparently) > funding, and (manifestly) professionally designed specifications for > LSID concerns that I am horrified at the prospect of adopting anything > else if LSID comes even close to being what the community needs. I can certainly understand that perspective, and that's one of the main reasons I am still semi-supportive of the LSID approach (i.e., existing code). My major concern has to do with the "Authority"/"Resolver" domain portion of the LSID, and the need (or non-need) for it to be an active, accessible domain in order to resolve the LSID. I'm also VERY concerned about there ever being a temptation to change an ID for an object (e.g., a specimen given from one Museum to another) -- unless it is understood that the non-"ObjectID" portion is really thought of as metadata of sorts, and ObjectID itself is globally unique by itself. I'll need to read the LSID spec in more detail, and give it some more thought, before I comment further on this. > Or, > let's forget about LSID and instead of deploying what satisfies 98% of > the needs in six months, we could roll our own and deploy what satisfies > 80% of the needs in a few years... If that were really the balance, then it would be a no-brainer. My concern would be adopting (effectively committing to) a scheme that satisfies 80% of the needs in six months, instead of being patient and picking a system that accomodates 98% of the need a few years from now. If I had confidence that we could implement LSIDs in a "test-drive" mode for a couple of years, without being fully committed to them, I'd be much more comfortable. But as someone who spends a considerable amount of time trying to undo the damage of "legacy" solutions to data problems that were hastily conceived, I'm trying to be cautious. > The design goals of TCP/IP and DNS, and their implementation, intersect > the requirements of Bio UUIDs only in a very small set, in fact, deep > down perhaps not at all. > > These protocols and the associated address syntax were designed > primarily for /routing/, not in any way designed to guarantee that a > datum twice received has any connection between the two occurrences. > > IP addresses are in no way persistent. > > IP addresses are not globally unique, albeit in several small and varied > ways: I don't think anyone (in this thread) was suggesting actually *USING* TCP/IP and DNS for BioGUIDs (at least I wasn't). Rather, I was looking to it as a source of ground-truthed schemes for reliably managing globally distributed information. For instance, would DNS synchronization/propagation serve as a useful model for gobally distributed, synchronized taxonomic registries? Or would the taxonomic registry work more effectively with one or a few centralized "masters" with which a larger set of replicates kept synchronized? I also think, as I explained earlier, that the hierarchical approach with centralized block ID issuance and local application might be instructive to a bioscheme. > In fact, if the UUIDs are meant to be semantically opaque it matters not > one whit who or how these matters are settled. Can you elaborate on what you mean by "semantically opaque"? I think I understand -- but the last thing this thread needs is ambiguity about the meaning of terms (i.e., "opaque semantics".... :-) ) > Exceptions to that are > social, not technical. ("If you don't let me decide X, I am not going to > use your scheme". "OK, then you won't participate in its benefits. > That's fine with me") ...and as I said before, the real challenge in establishing universally adopted BioGUIDs is not going to be technical; it's going to be social/political. > >>So, there is a heirarchy of how the "unique identifiers" are managed. > > > > There is > > > >>in fact a central authority, but it delegates to decentralized > > > > authorities. > > But this is mainly to distribute costs and speed issuance. It has > nothing to do with the naming scheme. The number of organizations to be > issued Bio GUIDs surely is several orders of magnitude less than those > to be issued IBv6 addresses. So I doubt any IPv6 issuance mechanisms are > instructive, at least in their purpose (and hence, if well implemented, > in their implementation). So are you saying that, because BioGUID traffic will be orders of magnitude smaller than internet domain traffic, there does not need to be delegation to decentralized authorities? If so, then we are in full agreement. > > GBIF seems to me to be the principle contender. > > I enthusiastically agree. Also the /principal/ contender. [Sorry, > couldn't resist. My fingers slip on that one sometimes too.] Ouch.... :-) > Not exactly. There is one scheme in case your application can't resolve > it in a more nearly "local" facility. There are /lots/ of ways to find > an IP address from a domain name. All those which comply fully with the > DNS protocol, however, can make available two pieces of metadata: the > TTL of the record it is offering, and the IP address of a machine at > which you can find an authoritative record of the assignment of the dns > name to the IP address. This protocol /might/, but you hope on > performance grounds usually /doesn't/, lead you up as far as the root > servers, and the "one scheme to bind them all". If there is any lesson > here at all, it is that name resolution protocols matter, but resolution > implementations don't. Yet another attribute on which, DNS/IP and LSID > are not distinguishable. This seems to be a fundamental point of confusion (for me anyway). Are the domain names embedded within LSIDs information-bearing in the sense that they are necessarily the internet domain at which the LSID is resolved? I guess I should read and understand Section 13.3 of the LSID spec before commenting further. > More often, only when the TTLs expire, there being no motivation to do > otherwise. O.K., there's a great analogy that may be useful if implementing a distributed system of synchronized/mirrored biological data servers: should they remain in synch at fixed time intervals? In real time with each data transaction? Or, should some sort of TTL feature be incorporated in data? Much to think about. But time for me to get some work done.... Aloha, Rich

1 0

Re: random notes on LSIDs
by Bob Morris 28 Sep '04

28 Sep '04

Rich No reading I can make of http://www.omg.org/cgi-bin/doc?dtc/2004-05-01 is consistent with the explanation offered below. "Authority" is not used at all in Section 9, "LSID Resolution Service". Instead, resolution is defined to be accomplished by a set of interfaces all of which take only a full (semantically opaque!!!! Sec. 8, p. 7!!!) LSID as argument. The interfaces correspond to methods offered by an LSID Resolution Service. Nowhere in Section 9 is there any relationship mentioned between resolution and the "authority identification" that is part of the syntax of an LSID. That is discussed in Section 8, "LSID Syntax" which carries the sentences (bottom of p. 7) that I have previously cited: "The authority identification is usually an Internet domain name. In this case it is recommended that it be owned by the organization that assigns an LSID in question". There are too many ">"s for me to understand who is claiming the stuff below, but whoever it is could do me a favor by telling me which part of the spec they are reading. (Or if they found a later document at OMG). Well, OK, I haven't yet read the "Accompanied Files" listed in Appendix A., some of which are normative and take precedence over the main document. Maybe the explanation comes from one of them. Richard Pyle wrote: > Many thanks for jumping in on this, Dave! > > >>As I think Dave V. and others have pointed out, when an LSID is resolved, >>DNS is used to find the LSID authority. The LSID authority then provides >>information about how the LSID can be served up (e.g. HTTP, SOAP, FTP), >>and where to get the data behind the LSID and associated metadata. If I >>start serving up LSIDs with the authority learningsite.com and later >>decide that I'm sick of serving up LSIDs, somebody else can take over >>serving up the data and the metadata. However, I (or they) still bear the >> responsibility of running the authority which points to the data. If my >>lsids have an authority like lsid.learningsite.com >>(urn:lsid:lsid.learningsite.com:foo:bar) then someone else can take over >>the authority by taking over lsid.learningsite.com and I can still have >>www.learningsite.com, mail.learningsite.com, etc... for myself. So, with a >>little planning, it's not so hard to deal with an authority going away as >>long as the people running it are responsible. >

1 0

Re: random notes on LSIDs
by dave thau 28 Sep '04

28 Sep '04

Hi Bob, Those are my >>s so I'll respond. Below I'm talking about the stuff in section 13.3 of the spec. Primarily page 27. Dave > Rich > > No reading I can make of http://www.omg.org/cgi-bin/doc?dtc/2004-05-01 > is consistent with the explanation offered below. "Authority" is not > used at all in Section 9, "LSID Resolution Service". Instead, resolution > is defined to be accomplished by a set of interfaces all of which take > only a full (semantically opaque!!!! Sec. 8, p. 7!!!) LSID as argument. > The interfaces correspond to methods offered by an LSID Resolution > Service. Nowhere in Section 9 is there any relationship mentioned > between resolution and the "authority identification" that is part of > the syntax of an LSID. That is discussed in Section 8, "LSID Syntax" > which carries the sentences (bottom of p. 7) that I have previously > cited: "The authority identification is usually an Internet domain name. > In this case it is recommended that it be owned by the organization that > assigns an LSID in question". > > There are too many ">"s for me to understand who is claiming the stuff > below, but whoever it is could do me a favor by telling me which part of > the spec they are reading. (Or if they found a later document at OMG). > Well, OK, I haven't yet read the "Accompanied Files" listed in Appendix > A., some of which are normative and take precedence over the main > document. Maybe the explanation comes from one of them. > > Richard Pyle wrote: > >> Many thanks for jumping in on this, Dave! >> >> >>>As I think Dave V. and others have pointed out, when an LSID is >>> resolved, >>>DNS is used to find the LSID authority. The LSID authority then >>> provides >>>information about how the LSID can be served up (e.g. HTTP, SOAP, FTP), >>>and where to get the data behind the LSID and associated metadata. If >>> I >>>start serving up LSIDs with the authority learningsite.com and later >>>decide that I'm sick of serving up LSIDs, somebody else can take over >>>serving up the data and the metadata. However, I (or they) still bear >>> the >>> responsibility of running the authority which points to the data. If my >>>lsids have an authority like lsid.learningsite.com >>>(urn:lsid:lsid.learningsite.com:foo:bar) then someone else can take over >>>the authority by taking over lsid.learningsite.com and I can still have >>>www.learningsite.com, mail.learningsite.com, etc... for myself. So, with >>> a >>>little planning, it's not so hard to deal with an authority going away >>> as >>>long as the people running it are responsible. >> > >

1 0

Re: BioGUIDs and the Internet Analogy
by Bob Morris 27 Sep '04

27 Sep '04

[For me, the bottom line---which however I nowhere state below---is: There is /so much/ existing free infrastructure source code---e.g. http://www-124.ibm.com/developerworks/oss/lsid/--- and (apparently) funding, and (manifestly) professionally designed specifications for LSID concerns that I am horrified at the prospect of adopting anything else if LSID comes even close to being what the community needs. Or, let's forget about LSID and instead of deploying what satisfies 98% of the needs in six months, we could roll our own and deploy what satisfies 80% of the needs in a few years... To my mind, the only question is this: is LSID good enough? So: first make the requirements. Then examine existing solutions. Arguing by analogy can lead to the "hammer" solution, usually attributed to Mark Twain: When all you own is a hammer, every problem begins to look like a nail. (Can someone give me an actual /reference/ to that quote????] ------------------------- The idea of modeling taxonomic uuids on the internet has been around for about 10 years, and has been written about explicitly for at least 3-4 by Hannu Saarenma and probably others. See his article in http://reports.eea.eu.int/technical_report_2001_70/en/Technical%20Report%20… The Tree of Life http://www.tolweb.org is a defacto such model, though without name authorities or persistence. The design goals of TCP/IP and DNS, and their implementation, intersect the requirements of Bio UUIDs only in a very small set, in fact, deep down perhaps not at all. These protocols and the associated address syntax were designed primarily for /routing/, not in any way designed to guarantee that a datum twice received has any connection between the two occurrences. IP addresses are in no way persistent. IP addresses are not globally unique, albeit in several small and varied ways: - The "private address blocks" 10.X.Y.Z, 172.16.0-0-172.31.255.255, and 192.168.X.Y may be assigned to any machine your and my network administrator care to, as long as neither is served on "the Internet", nor have an duplication on "an internet". -every machine implementing IP, besides whatever other addresses it may have, is always known to itself as 127.0.0.1; - the addresses in some IP address ranges are reserved to designate /many/ machines (multicast IP)). In general the design of the IP "nomenclature"---i.e. the IP addresses---is designed to solve routing problems, not identification problems. IP address syntax is intentionally not "semantically opaque", contrary to a requirement (well, a "should be") of LSIDs and well it should be. See Section 8 of http://www.omg.org/cgi-bin/doc?dtc/04-05-01 [If you don't see why IP addressing is not semantically opaque, try this exercise: With a single machine instruction (on most machines) how can you determine whether or not an IP address is in the Class B private address space(the 172 stuff above)? Sysadmin's, C programmers, and their families and employees are not eligible for this competition] In turn, DNS protocols are designed only to aid discovery of IP addresses. DNS addresses are also far from persistent (in fact, DNS records held anywhere have a "Time To Live" field which must be counted down until expiration, at which time the holder must acquire a new instance of the assignment of an IP address to a domain name.). [More below, interspersed] Richard Pyle wrote: >>Perhaps it would be useful to look at the issues being discussed about >>a bio identifier/locator/GUID in comparison to the same things that are >>needed for Internet communications. > > > I've long thought that parts of the DNS system would be extremely useful to > emulate in some aspects of bioinformatics data management (particularly > taxonomic names; see below). > > >>IP addresses have to be unique world-wide to make the Internet work. >>The Internet Corporation for Assigned Names and Numbers (ICANN- > > www.icann.org) > >>provides that uniqueness by assigning all the IP numbers in unique blocks > > or > >>ranges of numbers to "Internet Registries". > > > ...exactly the way that I envision an organization like GBIF would be > charged with the task of issuing UIDs for certain biological objects. > > >>There are Regional, National and Local Internet Registries that subdivide > > and > >>"license" IP addresses to ISPs, who in turn license IP addresses to > > organizations. Ah, this is accurate mainly for IPv6, which is much less chaotic than IPv4 (most of the current Internet) and in turn from the nearly formless void that was IPv2. But again, ultimate "licensees" do not get persistent IP addresses. In the US, virtually all dialup users get a different IP every time they connect, and most home broadband users only accidently keep their IP addresses, and only if they don't disconnect very long from the network. It's well worth comparing the design goals of IPv6 as articulated in http://www.apnic.net/docs/policy/ipv6-address-policy.html with those of LSID as articulated in Section 8 of the Draft Final Specification http://www.omg.org/cgi-bin/doc?dtc/04-05-01 > > There could be a useful analog for this in bioinformatics (particularly in > terms of individual institutions serving as regional registries for specimen > UIDs, or IC_N Commissions serving as "regional" registries for taxon name > UID assignment) -- but there doesn't necessarily have to be. > In fact, if the UUIDs are meant to be semantically opaque it matters not one whit who or how these matters are settled. Exceptions to that are social, not technical. ("If you don't let me decide X, I am not going to use your scheme". "OK, then you won't participate in its benefits. That's fine with me") If you want to see another example of lack of semantic opacity, read the ISBN standard ISO/TC 46/SC 9 N 326. Part of certain ISBNs can help you determine an allegedly common publishing-germaine attribute of the US, Zimbabwe, Puerto Rico, Ireland, Swaziland, part of Canada, and a few other "regions". > >>So, there is a heirarchy of how the "unique identifiers" are managed. > > There is > >>in fact a central authority, but it delegates to decentralized > > authorities. But this is mainly to distribute costs and speed issuance. It has nothing to do with the naming scheme. The number of organizations to be issued Bio GUIDs surely is several orders of magnitude less than those to be issued IBv6 addresses. So I doubt any IPv6 issuance mechanisms are instructive, at least in their purpose (and hence, if well implemented, in their implementation). > > To emulate this in bioinformatics, the "hierarchy" would be achieved simply > by allowing block-assignment of UIDs to various players -- but the important > point here is that only *one* organization ensures uniqueness (in the case > of Internet, of ISPs). The data to which those UIDs apply would be, for the > most part, the responsibility of the UID recipient, not the UID issuer (in > my world view). Thus: centralized issuance; delegated application. > > >>Is there an analogy for BioGUIDs to have a central body who divvies out > > the > >>unique numbers (like IP addresses) to decentralized bodies or large > > organizations? The International ISBN Organization http://www.isbn-international.org/ is roughly the IPv6 model. > > GBIF seems to me to be the principle contender. I enthusiastically agree. Also the /principal/ contender. [Sorry, couldn't resist. My fingers slip on that one sometimes too.] > > >>Since IP addresses are hard to memorize (and so too would be a BioGUID), > > "domain names" > >>are used. Starting with a domain name, you can first find the name and/or > > IP address > >>of a device, called the Domain Name Server, that can locate the IP address > > of other > >>computers. This is a form of indirect addressing. ICANN also manages the > > top-level > >>namespace for the Internet. They decide what the valid domain "extensions" > > are (like > >>.com, .uk) so that everybody, everywhere knows where to look them up. > > Then, the domain > >>name extensions are separated among the Regional, National, and Local > > Interent Registries > >>around the world. There is a scheme for where to find the IP addresses > > for every domain > >>extension (e.g. .com is on the ARIN registry, .com.uk is on the ). Not exactly. There is one scheme in case your application can't resolve it in a more nearly "local" facility. There are /lots/ of ways to find an IP address from a domain name. All those which comply fully with the DNS protocol, however, can make available two pieces of metadata: the TTL of the record it is offering, and the IP address of a machine at which you can find an authoritative record of the assignment of the dns name to the IP address. This protocol /might/, but you hope on performance grounds usually /doesn't/, lead you up as far as the root servers, and the "one scheme to bind them all". If there is any lesson here at all, it is that name resolution protocols matter, but resolution implementations don't. Yet another attribute on which, DNS/IP and LSID are not distinguishable. >>Then there is a layer of Domain Registrars who have been accredited by > > ICANN to assign > >>domain names for the domain extensions - e.g. tdwg.org. >>The domain name registrars are told by the owner of the domain where to > > find their particular > >>Domain Name Servers which may be many to enable redundancy - Primary, > > Secondary, Tertiary, > >>etc. Not quite. Normally, the registrant tells the registrar who has agreed to be the servers. The other case sometimes happens with "retailers" who are selling individuals domain names and ISP services at the same time. These redundant Domain Name Servers synchronize with each other at > > particular times > >>of day and may be located all around the world. More often, only when the TTLs expire, there being no motivation to do otherwise. >They are the main > > "switchboard" for a > >>particular organizations computer names and associated IP addresses. >>Then the individual organization can create multiple computers for the > > domain name - e.g. > >>www.tdwg.org - and add them to the Domain Name Server listing. There can > > be many computers > >>for a domain, for instance: info.tdwg.org, www2.tdwg.org, myname.tdwg.org. > > Each of these > >>can be a different computer with a different IP address. The redundant > > Domain Name Servers > >>all contain the list of all these names and what IP addresses they are. Not usually. The primary and secondary name servers would normally only cache tdwg.org permanently. They might /acquire/ a record for www.tdwg.org in response to a request, but they would not in general renew it after it expired and maybe not even keep it that long. To do so would be hideously unscalable. If I put 10,000 machines in my domain, my primary and secondary would be mighty unhappy if they had to keep them all cached. > > > This is analogous in many ways to how I would envision a global taxonomic > name service. UIDs are assigned by a centralized body (e.g., GBIF; or by > the IC_N Commissions) to individual names. Analogous to multiple redundant > Domain Name Servers (DNS) would be Taxon Name Servers (TNS). Rather than > administered by one organization (e.g., GBIF, ITIS, Species 2000, uBio, > etc.) these TSNs would be replicated on dozens or hundreds of servers all > over the world, and maintained as synchronized within some reasonable time > unit. Changes to any one replicate would be automatically propagated to all > replicates (either chaotically, or more strictly through one or a few > defined "hubs"). Instead of Domain names as surrogates for IP addresses, > there would be fully qualified "Basionyms" (e.g., > "OriginalGenusName.OriginalSpeciesName.OriginalAuthor.DescriptionYear.Page.O > therOriginalCitationDetailsAsNeeded") representations of the > less-human-friendly GUIDs (analogues to IP addresses). Ideally, this system > wouldn't be limited to just taxonomic names, but extended to all taxonomic > concepts, so that the "Domain Name" analogue would be extended to something > like: > > "OriginalGenusName.OriginalSpeciesName.OriginalAuthor.DescriptionYear.Page.O > therOriginalCitationDetailsAsNeeded_AppliedGenusName.AppliedSpeciesSpelling. > ConceptAuthor.ConceptYear.Page.OtherConceptCitationDetailsAsNeeded" > > >>The players in the Internet networking fabric all now play by these > > layered rules. > >>They all know them and follow them in order to keep the Internet running. > > This > >>stuff happens out of sight to everyone but the networking people and we > > all take it > >>for granted and assume it is simple. But, it's invisible not because it's > > simple, > >>but rather because it's disciplined. Agreed. Yet one more attribute where IP/DNS and LSID are not distinguishable. > > > Excellent synopsis, and (in my opinion), and excellent model to follow for > at least taxonomic names/concepts data. Perhaps also for specimen data (but > seems less intuitive for that.) This comes back to my earlier question about > whether it is vital that all bioinformatics GUIDs be of the same scheme; or > whether different schemes might be optimal for different classes of objects. > > Aloha, > Rich > > Richard L. Pyle, PhD > Natural Sciences Database Coordinator, Bishop Museum > 1525 Bernice St., Honolulu, HI 96817 > Ph: (808)848-4115, Fax: (808)847-8252 > email: deepreef(a)bishopmuseum.org > http://www.bishopmuseum.org/bishop/HBS/pylerichard.html

1 0

Re: Globally Unique Identifier & Donald Hobern's PPT
by Wouter Addink 27 Sep '04

27 Sep '04

Hi, I found the PowerPoint document from Donald really helpfull on some of the discussed issues. Unfortunately referring to it is a little difficult because the pages does not contain a unique identier :) But first some real-world experience with the 'GUID' used today in GBIF specimen network. The combination InstitutionCode, CollectionCode and CatalogNumber was chosen for that. Problems experienced last year: -the 'GUID combination' is not enforced and therefore not always used -Some collections belong to 2 or more Institutions or to none -If part of the collection moves to another institute, the guid combination is changed for that part. -The InstitutionCode should be unique, and providers where asking what to do if the code they wanted to use was already chosen, and who decides which institute may use an institutioncode if two institutes want to use it. There is no body responsible for that and there are no rules: the first Institute can claim a code, or the biggest or the most well known?? -In different science areas different InstitutionCodes within one Organisation where in use, which one to choose. -This 'GUID' can only be used for specimen, not for other life science objects. Now let's look at LSID syntax: urn:lsid:authority:namespace:object_identifier (:revision_number) About the first part; authority: It is naturally to want this to be unique. Therefore we can expect the same problems as mentioned above, plus unclearity about the difference issuing_authority vs. current_authority for the data. The problems with authority are important for the involved authorities only, not for the rest of the life science community. So discussions about it and establishing an authority that takes decisions in political conflicts are a waste of time. We can solve it by using a unique number only and maintaining a list that gives information about each number. It should be clear that this are only the initial issuing authority/authorities. About the second part; namespace: Things like 'Specimen' or 'Experiment'. In contrast with the first part, problems with this part are interesting for the whole lifescience community because applications will want to use this to decide whether the data can be used for a specific application. Standardisation of namespaces is necessary. I think it should be devided in two parts (not currently present in LSID) like MIME type image/jpeg etc: 'observation/abcd' or 'specimen/darwincore' for example. If we look at the Donalds PPT we see that in model 1: LSID assigned centrally, the namespace is chosen centrally and by model2: LSID assigned by each provider, the provider is free to choose one. Even language variations of naming a namespace can already give problems, so this is why I strongly favor a central mechanism here for assigning LSIDs, unless the provider is somehow forced to use a certain namespace class. The potential bottleneck problem is not really an issue I think (see also DNS mechanisms). If we choose central mechanism the issuing authority will always be GBIF (or do we need different authorities for different parts of Life Science?) so no problems with that also in this case. The third part; object id: no problems there. The last part; revision id: whether you need it depends: do you give the physical objects a GUID or the data records? With the first choice you do not need a revision number because the physical object will not change (or do they with living collections?). At first I thought that a GUID should be put on physical object: if you are looking for data, you are looking for data about a certain physical object, the source of the data is not (very) important. The same data elements in different sources about the same object should be equal, else there are errors. Donald's PPT gives the example that someone wants to refer to a LSID in a publication as a source. In that case you want to refer to a data source with a certain version. Then you need to give a GUID to the data and also you need revisions. Data is not persistent, it changes all the time. Giving a persistent identifier to it is very difficult and not many data systems have full revisions support. If a GUID for a 'physical object' is chosen, a thing like a species name or author name or country should not get a GUID. These are more a kind of attributes: most data will use one or more species names as 'metadata'. There needs to be central datasource for each of these 'metadata', like a NameBank for species names (with its own ID). I am not sure whether LSID was designed for a GUID to data or to physical objects. The use of namespace and object id instead of databasename and recordid seems to indicate that it was designed for physical objects, but why then the optional revision id? Instead of a revision id you can also assign a new GUID with every change, but then how to point to a new version from an old version of data (if you have the GUID of the old version, how to get the GUID of the new one). Requirements in Donald's PPT: -if a GUID is on a physical object, the GUID must not refer uniquely to a single data element, it must only be unique itself. It is also not a requirement in LSID specification. There will be overlap between the objects, so an object can belong to more then one IDs. For instance a researcher can have its own ID and also belong to the ID of the Institute he is working for. The data for overlapping elements like researcher name must be equal. -I would restrict the identifiers to life science objects. Issues to be resolved in Donald's PPT: It would be beneficial to maintain the GUID in the datasource itself (at least for the owner of the datasource), but not absolutely necessary. I see GUID in data records as a 'tightly coupled' model (which requires some work for existing databases). I can imagine also a 'loosely coupled' model where provider software is modified to get the identifier from a central server (or mirror). Wouter Addink

1 0

Re: BioGUIDs and the Internet Analogy
by Richard Pyle 27 Sep '04

27 Sep '04

> Perhaps it would be useful to look at the issues being discussed about > a bio identifier/locator/GUID in comparison to the same things that are > needed for Internet communications. I've long thought that parts of the DNS system would be extremely useful to emulate in some aspects of bioinformatics data management (particularly taxonomic names; see below). > IP addresses have to be unique world-wide to make the Internet work. > The Internet Corporation for Assigned Names and Numbers (ICANN- www.icann.org) > provides that uniqueness by assigning all the IP numbers in unique blocks or > ranges of numbers to "Internet Registries". ...exactly the way that I envision an organization like GBIF would be charged with the task of issuing UIDs for certain biological objects. > There are Regional, National and Local Internet Registries that subdivide and > "license" IP addresses to ISPs, who in turn license IP addresses to organizations. There could be a useful analog for this in bioinformatics (particularly in terms of individual institutions serving as regional registries for specimen UIDs, or IC_N Commissions serving as "regional" registries for taxon name UID assignment) -- but there doesn't necessarily have to be. > So, there is a heirarchy of how the "unique identifiers" are managed. There is > in fact a central authority, but it delegates to decentralized authorities. To emulate this in bioinformatics, the "hierarchy" would be achieved simply by allowing block-assignment of UIDs to various players -- but the important point here is that only *one* organization ensures uniqueness (in the case of Internet, of ISPs). The data to which those UIDs apply would be, for the most part, the responsibility of the UID recipient, not the UID issuer (in my world view). Thus: centralized issuance; delegated application. > Is there an analogy for BioGUIDs to have a central body who divvies out the > unique numbers (like IP addresses) to decentralized bodies or large organizations? GBIF seems to me to be the principle contender. > Since IP addresses are hard to memorize (and so too would be a BioGUID), "domain names" > are used. Starting with a domain name, you can first find the name and/or IP address > of a device, called the Domain Name Server, that can locate the IP address of other > computers. This is a form of indirect addressing. ICANN also manages the top-level > namespace for the Internet. They decide what the valid domain "extensions" are (like > .com, .uk) so that everybody, everywhere knows where to look them up. Then, the domain > name extensions are separated among the Regional, National, and Local Interent Registries > around the world. There is a scheme for where to find the IP addresses for every domain > extension (e.g. .com is on the ARIN registry, .com.uk is on the ). > Then there is a layer of Domain Registrars who have been accredited by ICANN to assign > domain names for the domain extensions - e.g. tdwg.org. > The domain name registrars are told by the owner of the domain where to find their particular > Domain Name Servers which may be many to enable redundancy - Primary, Secondary, Tertiary, > etc. These redundant Domain Name Servers synchronize with each other at particular times > of day and may be located all around the world. They are the main "switchboard" for a > particular organizations computer names and associated IP addresses. > Then the individual organization can create multiple computers for the domain name - e.g. > www.tdwg.org - and add them to the Domain Name Server listing. There can be many computers > for a domain, for instance: info.tdwg.org, www2.tdwg.org, myname.tdwg.org. Each of these > can be a different computer with a different IP address. The redundant Domain Name Servers > all contain the list of all these names and what IP addresses they are. This is analogous in many ways to how I would envision a global taxonomic name service. UIDs are assigned by a centralized body (e.g., GBIF; or by the IC_N Commissions) to individual names. Analogous to multiple redundant Domain Name Servers (DNS) would be Taxon Name Servers (TNS). Rather than administered by one organization (e.g., GBIF, ITIS, Species 2000, uBio, etc.) these TSNs would be replicated on dozens or hundreds of servers all over the world, and maintained as synchronized within some reasonable time unit. Changes to any one replicate would be automatically propagated to all replicates (either chaotically, or more strictly through one or a few defined "hubs"). Instead of Domain names as surrogates for IP addresses, there would be fully qualified "Basionyms" (e.g., "OriginalGenusName.OriginalSpeciesName.OriginalAuthor.DescriptionYear.Page.O therOriginalCitationDetailsAsNeeded") representations of the less-human-friendly GUIDs (analogues to IP addresses). Ideally, this system wouldn't be limited to just taxonomic names, but extended to all taxonomic concepts, so that the "Domain Name" analogue would be extended to something like: "OriginalGenusName.OriginalSpeciesName.OriginalAuthor.DescriptionYear.Page.O therOriginalCitationDetailsAsNeeded_AppliedGenusName.AppliedSpeciesSpelling. ConceptAuthor.ConceptYear.Page.OtherConceptCitationDetailsAsNeeded" > The players in the Internet networking fabric all now play by these layered rules. > They all know them and follow them in order to keep the Internet running. This > stuff happens out of sight to everyone but the networking people and we all take it > for granted and assume it is simple. But, it's invisible not because it's simple, > but rather because it's disciplined. Excellent synopsis, and (in my opinion), and excellent model to follow for at least taxonomic names/concepts data. Perhaps also for specimen data (but seems less intuitive for that.) This comes back to my earlier question about whether it is vital that all bioinformatics GUIDs be of the same scheme; or whether different schemes might be optimal for different classes of objects. Aloha, Rich Richard L. Pyle, PhD Natural Sciences Database Coordinator, Bishop Museum 1525 Bernice St., Honolulu, HI 96817 Ph: (808)848-4115, Fax: (808)847-8252 email: deepreef(a)bishopmuseum.org http://www.bishopmuseum.org/bishop/HBS/pylerichard.html

1 0

Re: random notes on LSIDs
by Richard Pyle 27 Sep '04

27 Sep '04

Many thanks for jumping in on this, Dave! > As I think Dave V. and others have pointed out, when an LSID is resolved, > DNS is used to find the LSID authority. The LSID authority then provides > information about how the LSID can be served up (e.g. HTTP, SOAP, FTP), > and where to get the data behind the LSID and associated metadata. If I > start serving up LSIDs with the authority learningsite.com and later > decide that I'm sick of serving up LSIDs, somebody else can take over > serving up the data and the metadata. However, I (or they) still bear the > responsibility of running the authority which points to the data. If my > lsids have an authority like lsid.learningsite.com > (urn:lsid:lsid.learningsite.com:foo:bar) then someone else can take over > the authority by taking over lsid.learningsite.com and I can still have > www.learningsite.com, mail.learningsite.com, etc... for myself. So, with a > little planning, it's not so hard to deal with an authority going away as > long as the people running it are responsible. O.K., that clears up things a great deal -- but also reinfornces my concerns about LSIDs for specimen data. What happens when Bishop Museum sends a specimen (or an entire collection -- but not all of the collections) to Smithsonian? If Bishop Museum still maintained lsid.bishopmuseum.org for its other collections, then the specimen would presumably need a new LSID based on lsid.Smithsonian.gov. Is there a protocol for a request coming into lsid.bishopmuseum.org to be automatically re-routed to lsid.Smithsonian.gov for just those specimens flagged as transferred? If so, then I would feel (slightly) more comfortable with LSIDs if the issuing organizations would agree to use some sort of independently unique value for <ObjectID>, that that portion would be preserved along with the specimen object. Of course, there is also the problem I alluded to earlier, where a specimen object with a GUID is fractioned and in need of new IDs - but this is a general problem which will need to be dealt with no matter the GUID scheme that is adopted. > * data and metadata > > With LSIDs there's a big difference between the data and metadata of an > LSID - and I think this is going to be the biggest challenge in deciding > how to use them in our context. What's the data? What's the metadata? > With gene sequences, the datum is the sequence, the metadata are things > like contact information, who did the sequencing, taxonomic information > about the thing sequenced, etc. There's an LTER site using LSIDs for > their data sets. The LSID data is the data set itself, and the metadata > is what you'd expect - a description of the data set, who the > investigators were, that sort of thing. NCBI has pubmed LSIDs - they're > not serving up the articles yet, but there's associated metadata in there. > For these things the division between data and metadata is fairly clear. > However, what is the data for taxa? What is the metadata? VERY difficult question. I perceive taxa (names & concepts) as artificial constructs, without unambiguosly objective reality; and as such, everything (except the UID itself) is metadata. However, original descriptions of taxon names do have an (almost) unambiguosly objective reality, as do documented statements about taxonomic concepts to which those statements apply. But even still, most of the attributes we might think of data elements for objects like an original description of a taxcon name, could also be interpreted as metadata. > Here's another interesting thing about data and metadata in LSIDs. When > you issue an LSID you're promising the the DATA behind that LSID never > changes. What about typographical/transcriptional errors? Can they be corrected? > * client stack versus authority server > > The LSID folks provide two batches of code - an authority server, for > people who want to to serve up LSIDs themselves, and an LSID Client stack > - which can be used by organizations to provide access to their LSIDs > and/or proxy LSIDs provided by other organizations. It may make sense for > an organization like GBIF to build a service using the Client Stack to > support both their own LSIDs and those served by other organizations. The > Client Stack has a caching mechanism which supports expiration information > from the primary authority, so the primary authority can update where the > LSID may be resolved and metadata of that authority. Does the expiration apply to the domain, or to the individual object? In other words, can a defined set of ObjectID's within one domain's LSID pool be re-directed, without having to redirect all calls to that LSID domain? Thanks again for your very useful insights! Aloha, Rich

1 0

random notes on LSIDs
by dave thau 27 Sep '04

27 Sep '04

Hello everyone, Sorry to come into this late - and forgive me if I'm covering trodden ground here, I just re-joined the list and may have missed a few posts. I've been looking at various GUID systems for SEEK - primarily the Handle System underlying DOI, and LSIDs. It looks like I'll be giving an introduction to GUIDs and focusing on LSIDs at TDWG in New Zealand in two weeks. Judging by the conversation here, I think I'll be keeping my introduction brief to allow maximal time for discussion! There are a few things about LSIDs that I want to point out. First.... as someone has mentioned, some of my early ramblings on GUIDs can be found here: http://cvs.ecoinformatics.org/cvs/cvsweb.cgi/seek/projects/taxon/docs/guid/ Some of these files got munged in CVS, but I've fixed them. These files are over six months old, and between then and now I've become more of an LSID fan. So, if you do bother reading that stuff, realize it's old and outdated and incomplete in too many places. I'm going to write a revised document encompassing and expanding all of that and I'll post here when it's completed. Now for some miscellaneous points about LSIDs * What happens when an LSID Authority goes away? As I think Dave V. and others have pointed out, when an LSID is resolved, DNS is used to find the LSID authority. The LSID authority then provides information about how the LSID can be served up (e.g. HTTP, SOAP, FTP), and where to get the data behind the LSID and associated metadata. If I start serving up LSIDs with the authority learningsite.com and later decide that I'm sick of serving up LSIDs, somebody else can take over serving up the data and the metadata. However, I (or they) still bear the responsibility of running the authority which points to the data. If my lsids have an authority like lsid.learningsite.com (urn:lsid:lsid.learningsite.com:foo:bar) then someone else can take over the authority by taking over lsid.learningsite.com and I can still have www.learningsite.com, mail.learningsite.com, etc... for myself. So, with a little planning, it's not so hard to deal with an authority going away as long as the people running it are responsible. * data and metadata With LSIDs there's a big difference between the data and metadata of an LSID - and I think this is going to be the biggest challenge in deciding how to use them in our context. What's the data? What's the metadata? With gene sequences, the datum is the sequence, the metadata are things like contact information, who did the sequencing, taxonomic information about the thing sequenced, etc. There's an LTER site using LSIDs for their data sets. The LSID data is the data set itself, and the metadata is what you'd expect - a description of the data set, who the investigators were, that sort of thing. NCBI has pubmed LSIDs - they're not serving up the articles yet, but there's associated metadata in there. For these things the division between data and metadata is fairly clear. However, what is the data for taxa? What is the metadata? Here's another interesting thing about data and metadata in LSIDs. When you issue an LSID you're promising the the DATA behind that LSID never changes. Additionally, there's only one authority ultimately responsible for pointing to the data, and that never changes (although as above, someone else can take the authority over). However, for metadata, there are no such promises. Metadata can change. Furthermore, organizations other than the authority can provide metadata, as long as the authority agrees to it and adds them to a list of authorized metadata providers. I dont know if this is such a great idea, but its in the specification. So, what's the data? What's the metadata? This question applies to any GUID system, really - the Handle System has the same issues, but less clearly defined. As an aside - the Handle System is very robust and the fee schedule is probably circumventable. However, I think LSIDs are better suited to the direction biodiversity informatics is taking - using XML-based standards and standard internet protocols to share data. * client stack versus authority server The LSID folks provide two batches of code - an authority server, for people who want to to serve up LSIDs themselves, and an LSID Client stack - which can be used by organizations to provide access to their LSIDs and/or proxy LSIDs provided by other organizations. It may make sense for an organization like GBIF to build a service using the Client Stack to support both their own LSIDs and those served by other organizations. The Client Stack has a caching mechanism which supports expiration information from the primary authority, so the primary authority can update where the LSID may be resolved and metadata of that authority. In this model, GBIF, or someone else, could support both their own LSIDs, and the LSIDs of others. Furthermore, it could choose which authorities it was going to resolve, so people who wanted to be sure to get "just the good stuff" according to GBIF could use the GBIF service. In addition, it could perform the de-duplication service that several people have mentioned - trying to maintain a one LSID per data item mapping. * lsid namespaces and file formats I don't think the namespace part of the LSID (urn:lsid:authority:namespace:object:version) is intended to be semantically loaded except for the relevant lsid authority. There is a way in the metadata to state what format the data comes in. It's not the traditional text/javascript mime-type tag - instead the format is another LSID! For example the FASTA protein sequence file format is: urn:lsid:i3c.org:formats:fasta. Clients that understand LSIDs, like the Launchpad application, can be set to attach applications to LSID formats so that clicking on an LSID with a given format opens up an appropriate application. Sorry to be so scattershot - it's hard to come into the middle of a huge topic like this. Im glad to see all this discussion its going to make working on my talk much easier (I think) Dave

1 0

Re: Globally Unique Identifier & Donald Hobern's PPT
by Richard Pyle 27 Sep '04

27 Sep '04

> -the 'GUID combination' is not enforced and therefore not always used > -Some collections belong to 2 or more Institutions or to none > -If part of the collection moves to another institute, the guid > combination > is changed for that part. > -The InstitutionCode should be unique, and providers where asking > what to do > if the code they wanted to use was already chosen, and who decides which > institute may use an institutioncode if two institutes want to > use it. There > is no body responsible for that and there are no rules: the first > Institute > can claim a code, or the biggest or the most well known?? > -In different science areas different InstitutionCodes within one > Organisation where in use, which one to choose. > -This 'GUID' can only be used for specimen, not for other life science > objects. Wholeheartedly agree on all counts!!! That's why I still see it as a "soft" ID (even with enforcement of unique registered Institution Codes, and enforced uniqueness of CollectionCode+CatalogNumber within a single InstitutionCode). It's a stop-gap to solve some of the problems, until a real GUID system is up, running, and broadly adopted. > Now let's look at LSID syntax: > urn:lsid:authority:namespace:object_identifier (:revision_number) > About the first part; authority: > It is naturally to want this to be unique. Therefore we can > expect the same > problems as mentioned above, plus unclearity about the difference > issuing_authority vs. current_authority for the data. As to uniqueness, I think that's (part of) the point of using a URL, instead of just an institution name or abbreviation. URLs seem to be effectively unique. As to the confusion about "issuing_authority" vs. "current_authority", count me among the befuddled. My interpretation of Dave Vieglais' posts were that the "Authority" URL was assumed to be the URL where the GUID is resolved to the data it represents. But Bob Morris' posts suggest otherwise ("The authority name is the /issuing/ authority. It's an authority for the LSID, not for its resolution or the underlying data."). Perhaps I misunderstood Dave's post? My primary concern about LSIDs is that (I thought) the URL used for the "authority" portion of the LSID must be live, online, active, and perpetual in order to resolve the data. If this is not the case, (i.e., if, as Bob says, it is only intended to indicate the *issuer*, not the current authority), then my concerns about LSIDs are greatly reduced. > The problems with authority are important for the involved > authorities only, > not for the rest of the life science community. Agreed!! And further, the authority makes sense for "local" or "owned" data (e.g., specimens, and attributes thereof), but not for "public" data (e.g., taxa, and attributes thereof). > So discussions about it and > establishing an authority that takes decisions in political > conflicts are a waste of time. > We can solve it by using a unique number only and maintaining a list that > gives information about each number. It should be clear that this are only > the initial issuing authority/authorities. Agreed for sure on the last sentence! And if I interpret the penultimate sentence correctly, then full agreement there as well. I had started writing a response to Bob Morris' post last night, but it got too late so I didn't finish it. It included the following: *************************************************** Bob Morris wrote: > I find nothing in the LSID current(?) proposed recommendation > http://www.omg.org/cgi-bin/doc?lifesci/2003-12-02 that specifies that > the "object identification" part of an LSID may not be a string of hex > digits and dashes. (Though I continue to fail to see why people are so > in love with these). I'm not particularly enamored by MACs or hex strings over other forms of unique identifiers, but I WOULD like to see a protocol for bioinformatics LSID generation that decreased the possibility of duplicate <ObjectID> portions of the LSIDs from different issuers to effectively zero. Maybe there is no meaningful technical reason to do this -- but I can't help but feel that if LSIDs are later determined not to be ideal for bioinformatics purposes, it may prove to be DAMN useful that the <ObjectID> portion alone serves effectively as its own gUid (emphasis on "U" intentional), even stripped of the remaining LSID components. *************************************************** Wouter: Is this what you meant by using a unique number? I also agree with Wouter (in response to Kevin Richards' concern about single-server bottleneck) that this should not be a concern. I seriously doubt that the sum total of all Life Sciences data calls would ever even approach the level of calls that Google receives. True, there will likely not be the same sort of money backing a life sciences GUID system that is behind Google's server farm -- but even still, given the fundamental importance such a system would have to such a wide variety of people receiving such a large chunk of grant money, I can certainly imagine justifying money on the order of 6 or 7 figures ($/euro) for such a server system, and I imagine that would be ample money by today's technological benchmarks (or, more likely, the benchmark a few more cycles of Moore's Law hence, which is probably when a system of this sort would start to receive really high volume requests) to get a system capable of meeting the demand. I think the real bottlenecks are sociological (i.e., getting these damned fickle life science practitioners to agree on anything), not technological. > The last part; revision id: whether you need it depends: do you give the > physical objects a GUID or the data records? Agreed! Another excerpt from my attempted reply to Bob Morris: *************************************************** Bob Morris wrote: > This brings up a point I may have missed in this discussion: LSIDs are > designed to give an identifier to /data/ not to physical objects. This > probably means that a fully compliant use of LSIDs for "specimens" will > be assigned to specimen records, not to specimen objects. This is good, > because if a physical object is moved it presumably gets a new specimen > record, which record, perhaps, has some metadata that tells the curation > history of the underlying object, including its previous LSIDs. > Seems like the analog of taxonomic synonomy to me.... I disagree -- as I stated before, I strongly feel that the number should be applied to the physical *object* (or virtual representation thereof -- such as an original description), not the electronic data record. If Bishop Museum sends a specimen to Smithsonian, the specimen now curated at Smithsonian is not, to my mind, a synonym of its earlier life at Bishop Museum. Nor is its data. Both the specimen, and its associated data, should be considered as fixed into perpetuity. As more and more data are transferred from the physical to an electronic form, those data should be associated to the same GUID for the object -- not multiple versions of the electronic representation of data related to that object. If the ID *must* be tied to the data record, instead of the object, then I take back everything I said earlier about versioning. In the data-centric scenario, versioning becomes absolutely *vital*. Personally, I think trying to manage GUIDs as record identifiers, rather than object identifiers, would introduce unnecessary and excessive complexity. Biologists are interested in the objects. The data records are just a convenient mechanism of information conveyance -- not important entities unto themselves. This may not apply to all aspects of Life Sciences, but I think it should apply to objects we're discussing here (specimens, taxa, references, etc.). *************************************************** > With the first choice you do > not need a revision number because the physical object will not change (or > do they with living collections?). Living collections do change, and so to unvochered observations, and so do records representing populations (rather than specific physical organisms). And even preserved specimens change over time (tracking condition, preservation status, etc.). But I think the GUID should be fixed to physical object, and that whatever dynamic properties of that object are worth recording over time should be associated back to the physical object. Where things get more complicated is how to define the "object". To some collections, the unit of "Object" may be multi-taxon/multi-specimen (e.g., fossil), or single-taxon/multi-specimen (e.g., lot), or single-taxon/single-specimen (individual specimen), or single-taxon/partial-specimen (a part of a specimen, like a skeleton vs. a skin). Single "objects" of any of these sorts may be fractioned (e.g., Isotypes, or simply splitting up a multi-specimen lot to send out to different institutions). So, one important question in such cases is whether one of the "child" objects retains the GUID of the "parent" object, and new GUIDs are assigned only to the remaining "child" objects (the way Linnaean taxonomy works for taxonomic concepts, and the way most institutions deal with catalog numbers for specimens). Or, do *all* child objects receive new GUIDs, each referring back to an historical "parent" object that no longer exists? The temptation is to support the latter, but in this case, what of a specimen that partial deteriorates and a portion of it is destroyed, rather than sent to a different institution? Logically, the remaining specimen should be treated no differently than it would have if the deteriorated portion was instead sent to a different institution, rather than destroyed, and hence receive a new GUID. But that's starting to sound an awful lot like condition monitoring. Perhaps this distinction should be left to the discretion of the GUID issuer/Object owner on a case-by-case basis? (Yikes! Inconsistency!) Or, perhaps this is where versioning comes in (where the versions are actual object versions, not electronic data versions)? This seems like a more complicated problem than the ones we have been discussing so far. > If a GUID for a 'physical object' is > chosen, a thing like a species name or author name or country > should not get a GUID. I disagree. For species names, the GUID would apply to the name's original description/creation event. Metadata for such never change -- they can only be corrected. For author names, I would argue that the object to which the GUID is applied should be thought of as the *name* of the author as a virtual physical object; not the author as a physical object. Multiple AuthorName objects could be linked to each other via an Alias scheme, and/or tied to a common "Person" (which could either be a separate GUID namespace, or be defined as a set of linked AuthorName objects). In the context of biological objects, place descriptors of all sorts are really just surrogates to defined two(three?)-dimentional physical spaces. The GUIDs for such should primarily be established for the physical space, not the name or other descriptors applied to that space. The GEOnet Names Server (GNS; http://earth-info.nima.mil/gns/html/index.html) seems to me to be a useful model to follow. They have two ID numbers for each record. One is the "Unique Feature Identifier" (UFI): "A number which uniquely identifies the feature [=place].", and the other is the "Unique Name Identifier" (UNI): "A number which uniquely identifies a name.". The point is, these representations to which I think GUIDs should be applied are effectively permanent/persistent. Aloha, Rich Richard L. Pyle, PhD Natural Sciences Database Coordinator, Bishop Museum 1525 Bernice St., Honolulu, HI 96817 Ph: (808)848-4115, Fax: (808)847-8252 email: deepreef(a)bishopmuseum.org http://www.bishopmuseum.org/bishop/HBS/pylerichard.html

1 0