Hi Steve et al.,<div><br></div><div>I agree. This was one of the reasons that I setup TaxonConcept the way I did. It attempts to connect both the LOD entities and the foreign key based entities.</div><div><br></div><div>For example:</div>
<div><br></div><div>The Racoon <a href="http://lod.taxonconcept.org/ses/CTZ8z.html">http://lod.taxonconcept.org/ses/CTZ8z.html</a></div><div><br></div><div><a href="http://lod.taxonconcept.org/ses/CTZ8z.html"></a>Has links to many other URL's and URI's as well as the integer id's for:</div>
<div><br></div><div>EoL</div><div>NCBI</div><div>ITIS</div><div>BOLD</div><div><br></div><div>* For some of these it might be best to represent these as a one to many since there are often many names for each concept.</div>
<div><br></div><div>I have uBio ID's in GeoSpecies but I thought that would be eventually pulled in via the GNI.</div><div><br></div><div>I also have a small set of other foreign keys for things like the Hymenoptera name server, FishBase, Mushroom Observer and Tropicos.</div>
<div><br></div><div>Since these are specific to specific subsets of organisms, and came on later in my project I thought it would be best to use a separate RDF file to map to those.</div><div><br></div><div>For instance with Fishbase <a href="http://assets.taxonconcept.org/fb/index.rdf">http://assets.taxonconcept.org/fb/index.rdf</a></div>
<div><br></div><div><meta charset="utf-8"><div>Insects like this one <a href="http://lod.taxonconcept.org/ses/ICmLC.html">http://lod.taxonconcept.org/ses/ICmLC.html</a> also have the id for bugguide if it exists there and I have found it under the same name or a synonym.</div>
</div><div><br></div><div>Of the ~105,000 concepts I have about 47,000 with ITIS ID's. This may be useful for your plant list and I can send you a spreadsheet if that is easier.</div><div><br></div><div>Most of the plants also have the USDA Plants identifier. In fact you might be able to get the ITIS numbers via the USDA Plants Database.</div>
<div><br></div><div>I have come to realize that many other groups see the solution to data access is with a custom API, but this requires understanding and debugging your code for each API.</div><div><br></div><div>Once the data is available in RDF it is one API for everything. Some issues like what to call each field can be overcome by simply rewriting (converting) the RDF.</div>
<div><br></div><div>This is easy as long as you have equivalent semantics in the meaning of the field.</div><div><br></div><div>For instance, it does not really matter if this name is represented as </div><div><font class="Apple-style-span" face="'courier new', monospace"><br>
</font></div><div><font class="Apple-style-span" face="'courier new', monospace"><txn:hasScientificName>Procyon lotor</txn:hasScientificName></font> or <font class="Apple-style-span" face="'courier new', monospace"><dwc:scientificName>Procyon lotor</<meta charset="utf-8">dwc:scientificName></font> </div>
<div><br></div><div>The important thing to understand is that in my model this field does not include the authorship string.</div><div><br></div><div>This makes it easier to map this to other datasets and publications that don't include the authorship string.</div>
<div><br></div><div><font class="Apple-style-span" face="'courier new', monospace"><txn:scientificNameAuthorship>(Linnaeus 1758)</txn:scientificNameAuthorship> </font></div><div><font class="Apple-style-span" face="'courier new', monospace"><br>
</font></div><div><font class="Apple-style-span" face="arial, helvetica, sans-serif"> * The scientificNameAuthorship should eventually be mapped to a publication or a list of probable publications. It is too ambiguous.</font></div>
<meta charset="utf-8"><div><br></div><div>There was a debate about <font class="Apple-style-span" face="arial, helvetica, sans-serif"><scientificName></font> earlier on the list which seemed to go back and forth. </div>
<div><br></div><div>I got tired of rewriting my examples each time and decided to use my own vocabulary that works in my example queries and has fields that map as closely to dwc as possible.</div><meta charset="utf-8"><div>
<br></div><div>- Pete<br><br><div class="gmail_quote">On Tue, May 31, 2011 at 7:07 AM, Steve Baskauf <span dir="ltr"><<a href="mailto:steve.baskauf@vanderbilt.edu">steve.baskauf@vanderbilt.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div bgcolor="#ffffff" text="#000000">
I had actually written a response to this thread about a week ago in
which I tried to clarify why I wanted to connect the ITIS and uBio
identifiers. However, I decided that the email was too cynical and not
helpful, so I erased it. However, I think that a couple of the points
I had in that email probably should have been made, so I will try to
state them again in a more constructive manner. <br>
<br>
My reason for wanting to connect the uBio and ITIS identifiers really
had nothing to do with making use of any of the tools or services that
either group provides. Rather it has to do with my desire to follow
the best practices for GUIDs as laid out in the TDWG GUID Applicability
Statement (now an official standard). In particular, I have in mind
Recommendations 2 and 8, which I paraphrase here as: "make HTTP URIs
out of your identifiers" and "stop making up new identifiers when
somebody else already has one for the thing you are talking about". I
suppose Recommendation 10 should also be mentioned, which I paraphrase
as "provide RDF/XML to users that want it". <br>
<br>
I am actually using ITIS TSNs internally in my database. However, last
time I checked there were no GUIDs based on TSNs that met the
recommendations I've paraphrased above. (The ITIS website does mention
"LSIDs" in the context of web services, but they don't follow either
recommendation 2 or 10.) However outdated they are, uBio identifiers
do actually meet recommendations 2 and 10 and that is why I wanted to
use them (although the http proxied forms are unnecessarily ugly and
long). So that explains in a nutshell the reason for my request. If
ITIS would provide a simple http URI form of their TSNs which could
resolve via content negotiation to either HTML or RDF/XML, it would be
much easier for me to just use them.<br>
<br>
OK, here is where I risk stepping on people's toes. So I'll try to
stomp gently. I think that the area of taxon names is one where the
TDWG community fails miserably at recommendation 8. I've lost count of
the number of different kinds of identifiers that are available for
referring to taxon names (this issue was discussed previously in the
thread that starts with
<a href="http://lists.tdwg.org/pipermail/tdwg-content/2011-January/002231.html" target="_blank">http://lists.tdwg.org/pipermail/tdwg-content/2011-January/002231.html</a>
so I won't repeat it here). I don't know about (nor particularly care
about) "turf" in this area, but I would challenge the community to get
serious about recommendation 8 and come up with some consensus about a
single, universal set of GUIDs for taxon names. Those identifiers
should (in my opinion which stems from the GUID recommendations):<br>
- be http URIs (rec 2)<br>
- be based on an existing identifier (rec 8)<br>
- return RDF/XML when a client requests it (rec 10)<br>
- not change (rec 4)<br>
I do not like proxied LSIDs (unnecessarily long with many useless
characters) and I despise UUIDs (what is the point of creating a long,
un-typeable string to replace a serial number that is already globally
unique if appended to a domain name?). Why not just register something
like <a href="http://purl.org/tn/" target="_blank">"http://purl.org/tn/"</a> (with "tn" representing "taxon name") and
stick one of the existing serial numbers onto it? The domain name
would be "turf-neutral" and anybody (GBIF, TDWG, or another
organization) could manage the actual resolution through redirection
from that domain. Somebody else could take over the management of the
GUIDs if the first group got tired of it or ran out of money. The
result would be a short and simple URI like
<a href="http://purl.org/tn/12345" target="_blank">"http://purl.org/tn/12345"</a>. What would be wrong with that? This is
not rocket science and could be easily accomplished by a few tech-savvy
people if the will were there.<br><font color="#888888">
<br>
Steve</font><div><div></div><div class="h5"><br>
<br>
Nicolson, David wrote:
<blockquote type="cite">
<pre>Hi Steve (and Dave),
[NB: After having composed the email below, just before sending it, I re-read your initial email more carefully and realized that you said you already had the ITIS TSNs, and were looking to add the NamebankIDs! Doh! Well, in case you (or anyone else) is interested in methods of matching names to get TSNs, I'll go ahead and send this anyway. But do note the comments below about the ITIS "versions" and ongoing overhaul of the vascular plant data in ITIS!!! -Dave]
I noticed this just before leaving work last week, and was out yesterday, but I wanted to chime in on this. I'm glad the uBio tools are meeting your needs (they do have some cool stuff!), but it should be noted that those tools are using a static snapshot of ITIS data from January 2009, and we have added about 50,000 additional scientific names, and updated tens of thousands of names beyond that (most of that in the last 6 months, as the frequency of loads dropped off in 2009-2010 due to technical issues).
I also want to note that ITIS is right in the middle of a full update of the vascular plant data in ITIS, and we're loading updated families on a monthly basis... and at long last we are tackling all the leftover issues from several bulk loads from USDA PLANTS data that left unreconciled bits of ITIS' older vascular plant data in various confusing states... so it is a VAST improvement that is underway.
There are several options for bouncing your names off the current version of ITIS.
One is to automate a matching process using the live ITIS data, based on the existing ITIS Web Services. I am CC'ing Alan Hampson, our IT fellow who built the Web Services ( <a href="http://www.itis.gov/web_service.html" target="_blank">http://www.itis.gov/web_service.html</a> ), in case you'd like to follow up with him on that option. The advantage is that once you have a process in place it is completely self-serve and can always utilize the current ITIS data. If you have the resources to do this I think it would be greatly to your advantage to use this approach.
You can explore some ideas for client software to use the services at:
<a href="http://www.itis.gov/ws_develop.html" target="_blank">http://www.itis.gov/ws_develop.html</a>
And for more information on ITIS web services try
<a href="http://www.itis.gov/ws_description.html" target="_blank">http://www.itis.gov/ws_description.html</a>
<a href="http://www.itis.gov/ITISWebService.xml" target="_blank">http://www.itis.gov/ITISWebService.xml</a>
The ability to flag multiply-matched names (as you noted) should probably be considered, so that appropriate manual steps can be taken. This solution will allow you to take advantage of subsequent updates to ITIS with a minimum of additional effort, and given that the plant data are in the middle of a major overhaul, this bears consideration!
Another possibility is to grab a full snapshot of the ITIS data, and load it into a database so you can do what you wish. The obvious drawback is that it goes out of date, as with the ITIS snapshot uBio is currently using. But it puts you in the driver's seat re what to do & getting new versions of ITIS. Some general information about the full exports is in the following page, although conspicuously absent is any mention of the MySQL version which (assuming you have the free MySQL properly installed & configured) can be loaded with just a few clicks or a few command lines (depending on your platform):
<a href="http://www.itis.gov/ftp_download.html" target="_blank">http://www.itis.gov/ftp_download.html</a>
And the current ITIS data are all here for downloading:
<a href="http://www.itis.gov/downloads/" target="_blank">http://www.itis.gov/downloads/</a>
A third option, which I note with some trepidation, is the old "Compare Nomenclature/Taxonomy" function on the ITIS site:
<a href="http://www.itis.gov/taxmatch_ftp.html" target="_blank">http://www.itis.gov/taxmatch_ftp.html</a>
This is a VERY old function that we do plan on replacing (timeframe not yet certain), and it is vulnerable to timeouts, etc., which is why it notes to limit the number of names per pass. But with smaller chunks of names it does work quite well. The caveat is that I would make sure to choose the 4th option in Step 4, as it is at least aware (unlike the 3 other options) of multiply-matched name cases, and lists them separately at the bottom of the report. Just a bare listing of the scientific names, with the word "name" at the top, saved as plain text, is all that is needed for input.
A final option would be to ask someone at ITIS to handle the matching for you (leaving you to decide re the multiply-matched names). This might be simple from your end, but is suboptimal as it leaves you in the same position as you are now should you want or need to compare names again in the future (whether due to acquiring new names in your system, or wanting to check against a later updated version of ITIS), and it pulls someone here (probably me) off of the push to get more updates into ITIS. But in a pinch, I'm certainly willing to try to help you, should it come down to that! I would just ask that you seriously consider the web services option (in particular) or the others above first.
I hope this helps some. If you have already run all your matches against the old "ITIS" data via uBio then you might consider re-running (against the current ITIS data) at least the leftover names that you did not yet get matched. Let us know if you have questions (the <a href="mailto:itiswebmaster@itis.gov" target="_blank">itiswebmaster@itis.gov</a> address goes to myself and Alan and several others, so that might be the best bet for a follow-up unless you have a question specifically for me).
Regards,
Dave
David Nicolson
Data Development Coordinator, Integrated Taxonomic Information System
Biologist, USGS Core Science Systems, Biological Informatics Program
<a href="mailto:nicolsod@si.edu" target="_blank">nicolsod@si.edu</a> Office <a href="tel:202-633-2149" value="+12026332149" target="_blank">202-633-2149</a> Fax <a href="tel:202-786-2934" value="+12027862934" target="_blank">202-786-2934</a>
<a href="http://www.itis.gov/" target="_blank">http://www.itis.gov/</a>
<a href="http://www.cbif.gc.ca/itis/" target="_blank">http://www.cbif.gc.ca/itis/</a>
"Nihil sumas necesse est..."
-----Original Message-----
Date: Fri, 20 May 2011 05:42:03 -0500
From: Steve Baskauf <a href="mailto:steve.baskauf@vanderbilt.edu" target="_blank"><steve.baskauf@vanderbilt.edu></a>
Subject: Re: [tdwg-content] ITIS TSNID to uBio NamebankIDs mapping
To: "David Remsen (GBIF)" <a href="mailto:dremsen@gbif.org" target="_blank"><dremsen@gbif.org></a>
Cc: <a href="mailto:tdwg-content@lists.tdwg.org" target="_blank">"tdwg-content@lists.tdwg.org"</a> <a href="mailto:tdwg-content@lists.tdwg.org" target="_blank"><tdwg-content@lists.tdwg.org></a>
Message-ID: <a href="mailto:4DD6457B.2080204@vanderbilt.edu" target="_blank"><4DD6457B.2080204@vanderbilt.edu></a>
Content-Type: text/plain; charset="iso-8859-1"
Thanks, all, for the responses. The "Compare to ITIS" function does
just what I want. I did a test run of 1000 names and it worked like a
charm. I will need to do a little massaging because sometimes two or
more ITIS IDs come back for each uBio ID. But I can handle that.
Steve
David Remsen (GBIF) wrote:
</pre>
<blockquote type="cite">
<pre>Steve
Have you tried this?
<a href="http://www.ubio.org/clients/ITIS/index.php" target="_blank">http://www.ubio.org/clients/ITIS/index.php</a>
or this?
<a href="http://www.ubio.org/services/mapper/index2.php" target="_blank">http://www.ubio.org/services/mapper/index2.php</a>
All this ubio talk makes me think we were on to something. Worth a thought about adopting the new stnadrds and tools and making it really smooth.
DR
On 20 May 2011, at 04:46, Steve Baskauf wrote:
</pre>
<blockquote type="cite">
<pre>I have generated a csv spreadsheet of about 39 000 plant names for the
U.S. which has the ITIS TSNIDs for the names in a column. I would like
to have the uBio Namebank IDs in another column of the table. I have
been looking them up on the uBio website by typing in the names as I
need to know the IDs, but after doing about 300 of them, I'm getting
tired of it. Does anybody have a clever idea of a way to get the other
38 000 Namebank IDs without looking them up. I'm sure that it would be
possible to find this out because uBio gets names from ITIS. However, I
haven't seen any clues about how to do it in an automated fashion. I'm
guessing that there might be some way to use the uBio web services, but
if so, it isn't obvious and I probably don't have the skills to carry it
out anyway.
Any ideas?
Steve
--
Steven J. Baskauf, Ph.D., Senior Lecturer
Vanderbilt University Dept. of Biological Sciences
postal mail address:
VU Station B 351634
Nashville, TN 37235-1634, U.S.A.
delivery address:
2125 Stevenson Center
1161 21st Ave., S.
Nashville, TN 37235
office: 2128 Stevenson Center
phone: <a href="tel:%28615%29%20343-4582" value="+16153434582" target="_blank">(615) 343-4582</a>, fax: <a href="tel:%28615%29%20343-6707" value="+16153436707" target="_blank">(615) 343-6707</a>
<a href="http://bioimages.vanderbilt.edu" target="_blank">http://bioimages.vanderbilt.edu</a>
_______________________________________________
tdwg-content mailing list
<a href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a>
<a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a>
</pre>
</blockquote>
<pre>.
</pre>
</blockquote>
<pre> </pre>
</blockquote>
<br>
<pre cols="72">--
Steven J. Baskauf, Ph.D., Senior Lecturer
Vanderbilt University Dept. of Biological Sciences
postal mail address:
VU Station B 351634
Nashville, TN 37235-1634, U.S.A.
delivery address:
2125 Stevenson Center
1161 21st Ave., S.
Nashville, TN 37235
office: 2128 Stevenson Center
phone: <a href="tel:%28615%29%20343-4582" value="+16153434582" target="_blank">(615) 343-4582</a>, fax: <a href="tel:%28615%29%20343-6707" value="+16153436707" target="_blank">(615) 343-6707</a>
<a href="http://bioimages.vanderbilt.edu" target="_blank">http://bioimages.vanderbilt.edu</a>
</pre>
</div></div></div>
<br>_______________________________________________<br>
tdwg-content mailing list<br>
<a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
<a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br>
------------------------------------------------------------------------------------<br>Pete DeVries<br>Department of Entomology<br>University of Wisconsin - Madison<br>445 Russell Laboratories<br>1630 Linden Drive<br>Madison, WI 53706<br>
Email: <a href="mailto:pdevries@wisc.edu" target="_blank">pdevries@wisc.edu</a><br><a href="http://www.taxonconcept.org/" target="_blank">TaxonConcept</a> & <a href="http://about.geospecies.org/" target="_blank">GeoSpecies</a> Knowledge Bases<br>
A Semantic Web, <a href="http://linkeddata.org/" target="_blank">Linked Open Data</a> Project<br>--------------------------------------------------------------------------------------<br>
</div>