[tdwg-tag] Recent DiGIR service disruption

Roderic Page r.page at bio.gla.ac.uk
Wed Oct 8 13:52:50 CEST 2008

Dear Roger,

On 8 Oct 2008, at 10:42, Roger Hyam wrote:

> Thanks to Rod and DaveV for their work.
> I'd like to make one observation and concur with David Shorthouse on
> one point.
> 1) Looking down the list of providers many are exposed as IP addresses
> (which is bad), some are on domain names but not port 80 (which is
> slightly bad) and some are clearly machine names rather than names
> that one would associate with services. It appears to me that a
> proportion people are not using DNS robustly to allow services to be
> moved from machine to machine in a sensible way. So aside from the
> current issues DaveV has outline there is inherent brittleness in the
> way we are implementing these services. This may well be because
> people in large or small institutions do not have easy access to DNS
> configuration for their institutional domain even if they understand
> the importance of it. This has big implications for role out of LSIDs
> or any other technology dependent on configuration of DNS.

I agree that this is a big issue, and mucking with the DNS strikes me  
as the Achilles heel of easy adoption of LSIDs. That said, I think we  
desperately need GUIDs for specimens, if only to enable other  
approaches, such as mirroring data.

> 2) To concur with DavidS on the number of providers. Whilst
> developing  the Biodiversity Collections Index we worked through some
> lists of DiGIR providers at various portals to see if we were missing
> any 'real' collections. What struck us was the level of overlap and
> the fact that we weren't discovering much (if anything) new. Rod's
> page lists < 200 DiGIR providers (if I try and weed out the duplicates
> by eye).  BCI has around 5,000 physical collections listed. This seems
> like a small percentage of cover even if you take into account BioCASe
> and TAPIR providers. There is overlap between provider and they do not
> very often cover complete collections. On the other hand the more
> important collections may be more likely to be covered.

I've  had a few emails about how to update the list of DiGIR (and  
other providers), so the list will change at some point (and grow).  
But you are right, the degree of coverage is pretty low, and I'd  
suggest heavily biased towards terrestrial vertebrates (perhaps  
because of the relatively small collection sizes, and the culture of  
assigning identifiers to individual specimens).



> Sorry for the lengthy cross post.
> All the best,
> Roger
> On 8 Oct 2008, at 07:06, Dave Vieglais wrote:
>> Apologies for cross posting.
>> Hopefully, most of you did not notice this, but if you did, here's
>> what happened.  A problem with resolving the digir.net domain was
>> recently identified by John Wieczorek and possibly others.  This
>> problem would have affected most, if not all DiGIR providers  
>> currently
>> operating.  Once identified, the issue was promptly resolved, and
>> ongoing problems are unlikely.
>> This message provides further information about the problem, how it
>> was resolved, and how recurrence will be avoided to ensure continued
>> operation of the DiGIR infrastructure.
>> = Problem =
>> The DNS entry "digir.net" is an A record that must point to an IP
>> address.  For several years digir.net resolved to the ip address of
>> "" which is a Source Forge machine that is configured as
>> a VHOST for digir.net.  At some point in the last 48hrs,  
>> ceased to respond as a VHOST for digir.net.  This meant that any web
>> service that was expecting to retrieve content from digir.net (e.g. http://digir.net/schema/protocol/2003/1.0/digir.xsd
>> ) failed, with significant operational repercussions.  For technical
>> reasons, most other digir.net DNS entries were unaffected (e.g. www.digir.net
>> ).
>> = Resolution =
>> The new Source Forge VHOST servicing www.digir.net was identified as
>> "", and the DNS entry for digir.net was updated
>> appropriately.  Since the TTL on that DNS record was set relatively
>> short (600 seconds), the DNS infrastructure picked up the new
>> information fairly quickly, and testing indicates that digir.net URLs
>> are now being correctly resolved from several locations around the
>> world.
>> = What You Need To Do =
>> Nothing.  You should not notice any ongoing disruption.
>> = Ensuring Continued Operation =
>> There are three major issues that lead to this malfunction (#2 and #3
>> are informational only for developers of new services and tools):
>> 1. We have no control over Source Forge, and nor should digir.net
>> infrastructure be dependent on them
>> 2. Much of the existing installed DiGIR infrastructure references
>> "digir.net" directly and so DNS level indirection is not practical.
>> 3. HTTP redirects need to be supported in service implementations
>> and their clients.  I suspect that a lot of DiGIR software services
>> and clients do not follow redirects properly.
>> == Issue #1 ==
>> A DNS monitoring service is now in place that will update the DNS
>> entries if the source forge machine changes again.
>> Also, a replica of the DiGIR web material currently hosted by Source
>> Forge is being created on a machine that we have complete control  
>> over
>> and is located on a fast internet backbone.  The replica machine will
>> be monitored with a system that will provide automatic notification  
>> to
>> appropriate contacts in the event of failure.  This machine will be
>> located at the University of Kansas at least initially.  Once
>> operational, the DNS entry for digir.net will be modified to point to
>> this machine.
>> DNS services are handled by dyndns.org.  Since their inception they
>> have provided a 100% uptime for their clients which includes domains
>> such as CNN, Mozilla, and Twitter.  Since digir.net was an early
>> adopter of their services, the DNS service provided by DynDNS does  
>> not
>> expire.  DNS registration does expire however, and this relatively
>> minor expense is covered by the University of Kansas with assistance
>> from the National Science Foundation.
>> == Issue #2 ==
>> In retrospect, something like "http://schema.digir.net" rather than  
>> "http://digir.net
>> " should have been used.  This would have provided a low level (DNS)
>> mechanism for indirection that would have been unaffected by this
>> unanticipated hardware change.  Altering these entries in DiGIR
>> providers and tools is really not practical at this stage, so it is
>> unlikely that this issue can be directly addressed.
>> == Issue #3 ==
>> HTTP offers a mechanism for temporary and permanent redirection.  In
>> order to take advantage of this functionality, it is necessary for
>> HTTP clients to properly interpret the HTTP 301 and 307 status codes
>> and the respective response headers to determine the new location of
>> the resource being resolved.  This is the mechanism employed by PURLs
>> and TinyURL for example.  Unfortunately, many developers take short
>> cuts when building tools to retrieve content from URLs and avoid the
>> additional small amount of work necessary to handle redirect
>> messages.  Where ever possible, developers of services and clients
>> should use libraries that at least properly handle HTTP redirects
>> which will allow the use of PURLs and other mechanisms for providing
>> an additional level of indirection in URL resolution where necessary.
>> regards,
>>  Dave Vieglais
>> ---
>> Biodiversity Research Center
>> University of Kansas
>> _______________________________________________
>> tdwg-tag mailing list
>> tdwg-tag at lists.tdwg.org
>> http://lists.tdwg.org/mailman/listinfo/tdwg-tag
> -------------------------------------------------------------
> Roger Hyam
> Roger at BiodiversityCollectionsIndex.org
> http://www.BiodiversityCollectionsIndex.org
> -------------------------------------------------------------
> Royal Botanic Garden Edinburgh
> 20A Inverleith Row, Edinburgh, EH3 5LR, UK
> Tel: +44 131 552 7171 ext 3015
> Fax: +44 131 248 2901
> http://www.rbge.org.uk/
> -------------------------------------------------------------
> _______________________________________________
> tdwg-tag mailing list
> tdwg-tag at lists.tdwg.org
> http://lists.tdwg.org/mailman/listinfo/tdwg-tag

Roderic Page
Professor of Taxonomy
Graham Kerr Building
University of Glasgow
Glasgow G12 8QQ, UK

Email: r.page at bio.gla.ac.uk
Tel: +44 141 330 4778
Fax: +44 141 330 2792
AIM: rodpage1962 at aim.com
Facebook: http://www.facebook.com/profile.php?id=1112517192
Twitter: http://twitter.com/rdmpage
Blog: http://iphylo.blogspot.com
Home page: http://taxonomy.zoology.gla.ac.uk/rod/rod.html

More information about the tdwg-tag mailing list