<div dir="ltr"><div><font color="#500050" face="arial, sans-serif">I see the rational for enabling this in Darwin Core Archives and adding the new terms. </font><font color="#500050" face="arial, sans-serif">However, back to what Matt Jones brought up: "</font><span style="font-size:12.800000190734863px;color:rgb(80,0,80);font-family:arial,sans-serif">won't we just end up with a new syntax that does essentially what </span><span style="font-size:12.800000190734863px;color:rgb(80,0,80);font-family:arial,sans-serif">O&M</span><span style="font-size:12.800000190734863px;color:rgb(80,0,80);font-family:arial,sans-serif"> and OBOE do now?". </span></div>
<div><span style="color:rgb(80,0,80);font-family:arial,sans-serif;font-size:12.800000190734863px"><br></span></div><div><span style="color:rgb(80,0,80);font-family:arial,sans-serif;font-size:12.800000190734863px">We should include explicit references to existing terms/definitions that encapsulate what we're talking about, e.g. in our MaterialSample proposal last year we linked the an existing term in OBI, which has a much richer description and context for MaterialSample than what we considered (</span><font color="#500050" face="arial, sans-serif"><a href="https://code.google.com/p/darwincore/issues/detail?id=167">https://code.google.com/p/darwincore/issues/detail?id=167</a>) <br>
<br></font></div><div class="gmail_extra"><div class="gmail_quote">Have we explored the possibility of doing this with OBOE? I'm not suggesting we adopt OBOE wholesale, but it seems like we have a good opportunity to enable better semantic linking with that efforts.</div>
<div class="gmail_quote"><br></div><div class="gmail_quote">John</div><div class="gmail_quote"><br></div><div class="gmail_quote">On Thu, Aug 28, 2014 at 4:23 AM, Éamonn Ó Tuama [GBIF] <span dir="ltr"><<a href="mailto:eotuama@gbif.org" target="_blank">eotuama@gbif.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Thanks, Ramona and Rob.<br>
<br>
I'd like to add a few points following on Markus's reply.<br>
<br>
I think your pressing of the need for a robust semantic model for<br>
biodiversity sample/survey data is incontestable – we do need one and it<br>
should enable rich data integration once it is defined and the tools and<br>
data standards to support it become available. However, GBIF is faced with<br>
the immediate task of making sample-based data discoverable and accessible<br>
using its current ecosystem of tools (IPT) and exchange standards (DwC;<br>
EML). Waiting for a functional, implementable semantic model and the tools<br>
and support services for it is just not an option for us right now.<br>
<br>
We have already spend considerable time in analysing the merits of<br>
Occurrence core vs Event core and have opted for an Event core for reasons<br>
previously given. I don’t believe we are trying to reconfigure Event (“an<br>
action that occurs at a place and during a period of time”) and regardless<br>
of whether we use Occurrence or Event, the need for some additional terms<br>
arise (e.g., quantity, quantityType, samplingGeometry, samplingUnit). Once<br>
the BCO model is available for uptake, it should be possible to develop a<br>
mapping between it and the simple DwC sample model.<br>
<br>
So GBIF’s stance is that we need to take a two-pronged approach by exploring<br>
how the IPT and DwC-A can be adapted for publishing sample-based data in the<br>
near term while supporting the work of TDWG and groups such as the BCO in<br>
advancing biodiversity informatics. GBIF has already engaged in the work of<br>
the BCO and will continue to do so.<br>
<span class=""><font color="#888888"><br>
Éamonn<br>
</font></span><div class="im"><br>
-----Original Message-----<br>
From: <a href="mailto:tdwg-content-bounces@lists.tdwg.org">tdwg-content-bounces@lists.tdwg.org</a><br>
[mailto:<a href="mailto:tdwg-content-bounces@lists.tdwg.org">tdwg-content-bounces@lists.tdwg.org</a>] On Behalf Of Markus Döring<br>
Sent: 28 August 2014 12:44<br>
To: Ramona Walls<br>
Cc: TDWG Content Mailing List<br>
</div><div class=""><div class="h5">Subject: Re: [tdwg-content] tdwg-content Digest, Vol 63, Issue 6<br>
<br>
Hi Ramona & Rob,<br>
<br>
The Event proposal does not try to change the semantics of an Event, it just<br>
uses the existing Darwin Core Event "class" at the core in Darwin Core<br>
archives. The actual change proposed is simply adding 3 new terms to the<br>
Event "group" to better share information about sampling methods & efforts,<br>
extending the existing limited capabilities of Darwin Core which already has<br>
the terms dwc:samplingProtocol and dwc:samplingEffort. It also proposes 2<br>
new terms for dealing with quantity of Occurrences, something that has been<br>
discussed since 2012 now, when I had proposed a new abundance term [2].<br>
<br>
<br>
In general application of Darwin Core is not at all limited to specimens and<br>
observations. It is used for sharing taxonomic datasets already and it's<br>
definition and goal is broad. Let me cite some of the introduction to Darwin<br>
Core [1]:<br>
<br>
What is the Darwin Core?<br>
The Darwin Core is body of standards. It includes a glossary of terms (in<br>
other contexts these might be called properties, elements, fields, columns,<br>
attributes, or concepts) intended to facilitate the sharing of information<br>
about biological diversity by providing reference definitions, examples, and<br>
commentaries. The Darwin Core is primarily based on taxa, their occurrence<br>
in nature as documented by observations, specimens, samples, and related<br>
information.<br>
<br>
Motivation: The Darwin Core standard was originally conceived to facilitate<br>
the discovery, retrieval, and integration of information about modern<br>
biological specimens, their spatiotemporal occurrence, and their supporting<br>
evidence housed in collections (physical or digital). The Darwin Core today<br>
is broader in scope and more versatile. It is meant to provide a stable<br>
standard reference for sharing information on biological diversity. As a<br>
glossary of terms, the Darwin Core is meant to provide stable semantic<br>
definitions with the goal of being maximally reusable in a variety of<br>
contexts.<br>
<br>
<br>
Markus<br>
<br>
<br>
[1] <a href="http://rs.tdwg.org/dwc/index.htm" target="_blank">http://rs.tdwg.org/dwc/index.htm</a><br>
[2] <a href="https://code.google.com/p/darwincore/issues/detail?id=142" target="_blank">https://code.google.com/p/darwincore/issues/detail?id=142</a><br>
<br>
<br>
--<br>
Markus Döring<br>
Software Developer<br>
Global Biodiversity Information Facility (GBIF)<br>
<a href="mailto:mdoering@gbif.org">mdoering@gbif.org</a><br>
<a href="http://www.gbif.org" target="_blank">http://www.gbif.org</a><br>
<br>
<br>
<br>
<br>
<br>
On 27 Aug 2014, at 18:57, Ramona Walls <<a href="mailto:rlwalls2008@gmail.com">rlwalls2008@gmail.com</a>> wrote:<br>
<br>
> I think it is important to consider the purpose of both Darwin Core and<br>
DwC archives in deciding whether or not to expand them, but we should use<br>
that consideration to address the question at hand, which is whether or not<br>
to add an Event core and additional properties to describe events.<br>
><br>
> Describing the exchange format before the semantics is the wrong way to<br>
go, given that we now have a framework for developing semantics. Expanding<br>
Darwin Core before we adequately model survey data is bound to lead to<br>
problems later, when we try to retro-fit the semantics to Darwin Core Event<br>
archives. This is exactly the problem we are running into now with Occurance<br>
archives, and we have the opportunity to avoid it.<br>
><br>
> I suggest we first use existing ontologies to model survey data, then deal<br>
with if and how to exchange that information in DwC-A. This is what I was<br>
hinting at in my first email, but should have said more explicitly.<br>
><br>
> Ramona<br>
><br>
> ------------------------------------------------------<br>
> Ramona L. Walls, Ph.D.<br>
> Scientific Analyst, The iPlant Collaborative, University of Arizona<br>
> Research Associate, Bio5 Institute, University of Arizona<br>
> Laboratory Research Associate, New York Botanical Garden<br>
><br>
><br>
> On Wed, Aug 27, 2014 at 9:14 AM, Robert Guralnick<br>
<<a href="mailto:Robert.Guralnick@colorado.edu">Robert.Guralnick@colorado.edu</a>> wrote:<br>
><br>
> It may be a sensible view for Darwin Core Archives and their intended<br>
use, but Tim's email suggests we should be putting the method of delivery<br>
ahead of the standard that delivers that content. If this was just about<br>
DwC-As, why not develop a survey extension that links each occurrence to<br>
information about the survey process using the existing star-schema methods<br>
we have in place? Why are we discussing adding terms to the Darwin Core or<br>
trying to fully reconfigure what we call an Event? That is what is on the<br>
table, not DwC-As and how we use them. Or am I missing something?<br>
><br>
> Best, Rob<br>
><br>
><br>
><br>
> On Wed, Aug 27, 2014 at 10:01 AM, Ramona Walls <<a href="mailto:rlwalls2008@gmail.com">rlwalls2008@gmail.com</a>><br>
wrote:<br>
> Thanks, Tim, and yes, DwC-A as a view (but not necessarily the primary<br>
archive) of data seems like the right point of view.<br>
><br>
> Ramona<br>
><br>
> ------------------------------------------------------<br>
> Ramona L. Walls, Ph.D.<br>
> Scientific Analyst, The iPlant Collaborative, University of Arizona<br>
> Research Associate, Bio5 Institute, University of Arizona<br>
> Laboratory Research Associate, New York Botanical Garden<br>
><br>
><br>
> On Wed, Aug 27, 2014 at 1:58 AM, Tim Robertson <<a href="mailto:trobertson@gbif.org">trobertson@gbif.org</a>><br>
wrote:<br>
> Hi Ramona,<br>
><br>
> Those are good points, and I’d like to come back to the original thinking<br>
behind the DwC-A.<br>
><br>
> It was designed and intended to be a simple way of exposing a complete<br>
view of a dataset, primarily for building sophisticated indexes, inventories<br>
and allowing basic analytics (e.g. GBIF.org being one sophisticated index).<br>
We found that the star schema provided the flexibility to do a lot, and with<br>
the bundled metadata (e.g. EML) was enough to trace provenance and allow<br>
users to determine if the dataset might be fit for various uses. In many<br>
cases this represents the complete (e.g. lossless) view of a dataset.<br>
><br>
> What we are discussing here are far richer datasets, where shoe-horning<br>
content into the star schema becomes lossy for some, although we’re finding<br>
other cases where it is indeed lossless. I believe we should be looking to<br>
harmonise ontologies / models etc as you mention but in parallel we should<br>
define one or more star schema views that can still be used for discovery /<br>
reporting / basic analytical purpose, and not long term archival of the<br>
dataset. The dataset would then have the canonical rich form and an<br>
additional DwC-A view. What I write here is applicable to all content types<br>
of course.<br>
><br>
> Please also note that many people put supplementary files in the DwC-A<br>
which are ignored by DwC-A readers but could be a way of keeping the richer<br>
view in the bundle. If one wished you can describe those supplementary<br>
files in the EML document.<br>
><br>
> Does this gel with the view of others as well?<br>
><br>
> Cheers,<br>
> Tim<br>
><br>
><br>
><br>
> On 27 Aug 2014, at 02:55, Ramona Walls <<a href="mailto:rlwalls2008@gmail.com">rlwalls2008@gmail.com</a>> wrote:<br>
><br>
>> I think Matt hit the nail on the head. Although Darwin Core can be used<br>
to exchange survey data, it lacks the semantics and structure necessary to<br>
archive the data without loss of information. I think the biodiversity<br>
community would be better served devoting energy to harmonizing existing<br>
technologies such as OGC, OBOE, and BCO, not to mention the many database<br>
for storing plot or survey data. The goal should be to preserve the data in<br>
the most informative manner possible.<br>
>><br>
>> There is a strong a case for wanting to search across all evidence for<br>
occurences, including surveys and point occurences, so I can see possible<br>
demand for a tool that would extract occurences from survey data to a DwC<br>
archive. However, I am very concerned that making a DwC archive the primary<br>
exchange format for survey or plot data commits us to a path of losing<br>
information from the start, for all but the simplest sampling schemas.<br>
>><br>
>> Ramona<br>
>><br>
>> ------------------------------------------------------<br>
>> Ramona L. Walls, Ph.D.<br>
>> Scientific Analyst, The iPlant Collaborative, University of Arizona<br>
>> Research Associate, Bio5 Institute, University of Arizona<br>
>> Laboratory Research Associate, New York Botanical Garden<br>
>><br>
>><br>
>> On Fri, Aug 22, 2014 at 3:00 AM, <<a href="mailto:tdwg-content-request@lists.tdwg.org">tdwg-content-request@lists.tdwg.org</a>><br>
wrote:<br>
>> Send tdwg-content mailing list submissions to<br>
>> <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>><br>
>> To subscribe or unsubscribe via the World Wide Web, visit<br>
>> <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>> or, via email, send a message with subject or body 'help' to<br>
>> <a href="mailto:tdwg-content-request@lists.tdwg.org">tdwg-content-request@lists.tdwg.org</a><br>
>><br>
>> You can reach the person managing the list at<br>
>> <a href="mailto:tdwg-content-owner@lists.tdwg.org">tdwg-content-owner@lists.tdwg.org</a><br>
>><br>
>> When replying, please edit your Subject line so it is more specific<br>
>> than "Re: Contents of tdwg-content digest..."<br>
>><br>
>><br>
>> Today's Topics:<br>
>><br>
>> 1. Re: Darwin Core: proposed news terms for expressing sample<br>
>> data (Matt Jones)<br>
>> 2. Re: Darwin Core: proposed news terms for expressing sample<br>
>> data (Donald Hobern [GBIF])<br>
>><br>
>><br>
>> ----------------------------------------------------------------------<br>
>><br>
>> Message: 1<br>
>> Date: Thu, 21 Aug 2014 18:52:06 -0800<br>
>> From: Matt Jones <<a href="mailto:jones@nceas.ucsb.edu">jones@nceas.ucsb.edu</a>><br>
>> Subject: Re: [tdwg-content] Darwin Core: proposed news terms for<br>
>> expressing sample data<br>
>> To: ?amonn ? Tuama [GBIF] <<a href="mailto:eotuama@gbif.org">eotuama@gbif.org</a>><br>
>> Cc: TDWG Content Mailing List <<a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a>><br>
>> Message-ID:<br>
>><br>
<CAFSW8xkx7uRP9PC2g3=<a href="mailto:JT_VJanqujH8nPXoz8GXwh%2BJwKw5Ccw@mail.gmail.com">JT_VJanqujH8nPXoz8GXwh+JwKw5Ccw@mail.gmail.com</a>><br>
>> Content-Type: text/plain; charset="utf-8"<br>
>><br>
>> This proposal is treading on ground that is quite similar to other<br>
>> observations and measurements standards for data exchange that are<br>
already<br>
>> mature, in particular:<br>
>><br>
>> * OGC Observations and Measurements (<br>
>> <a href="http://www.opengeospatial.org/standards/om" target="_blank">http://www.opengeospatial.org/standards/om</a>)<br>
>> * Extensible Observation Ontology (OBOE;<br>
>> <a href="https://semtools.ecoinformatics.org/oboe" target="_blank">https://semtools.ecoinformatics.org/oboe</a>)<br>
>><br>
>> The former is a standard and broadly deployed, whereas the latter is part<br>
>> of a research program in the use of ontologies for measurements. Through<br>
>> collaboration between the two projects, they've been modified to be<br>
>> reasonably isomorphic, but O&M uses an XML serialization while OBOE uses<br>
an<br>
>> OWL-DL serialization. They largely express the same measurements and<br>
>> sampling model once one gets beyond the terminology differences.<br>
>><br>
>> So, I'm wondering if it make much sense to extend Darwin Core, which is<br>
at<br>
>> heart an Occurrence exchange syntax, into this measurements area that is<br>
>> well represented by these other existing specifications? I'm curious to<br>
>> hear why people would even want to do this. And if we do go down this<br>
>> path, won't we just end up with a new syntax that does essentially what<br>
O&M<br>
>> and OBOE do now?<br>
>><br>
>> Matt<br>
>><br>
>><br>
>><br>
>> On Thu, Aug 21, 2014 at 12:22 AM, ?amonn ? Tuama [GBIF]<br>
<<a href="mailto:eotuama@gbif.org">eotuama@gbif.org</a>><br>
>> wrote:<br>
>><br>
>> > Hi Rob, Anne, Rich,<br>
>> ><br>
>> ><br>
>> ><br>
>> > I think Markus has answered your question as to why we opted for an<br>
Event<br>
>> > core which is being used in the sense described by Anne and Rich. For<br>
any<br>
>> > event, you can have a list of species in an Occurrence extension and<br>
for<br>
>> > each species, you can include quantity and quantityType, e.g., biomass,<br>
>> > etc. The proposed term eventSeriesID was intended for linking together<br>
>> > related events, although it now looks like parentEventID might be a<br>
better,<br>
>> > more flexible term. The measurementOrFact extension is a good fit for<br>
>> > capturing environmental information relating to an event. See, e.g.,<br>
the<br>
>> > Gialova Lagoon brackish water invertebrate test data set [1] where a<br>
set<br>
>> > of 18 environmental variables, including temp, pH, Rdx, particulate<br>
organic<br>
>> > matter, dissolved oxygen, salinity, chlorophyll-a were measured for<br>
each<br>
>> > sampling station-sampling period combination. An example mapping is:<br>
>> ><br>
>> ><br>
>> ><br>
>> > Id measurementType measurementValue<br>
>> > measurementUnit measurementRemarks<br>
>> ><br>
>> > IA Tmp (sed) 21.5<br>
>> > degree C Tmp<br>
>> > (sed): temperature at the bottom surface<br>
>> ><br>
>> ><br>
>> ><br>
>> > **Controlled vocabularies**<br>
>> ><br>
>> > Ideally, the values for samplingUnit and quantityType would be selected<br>
>> > from controlled vocabularies. This is, effectively, what we do by<br>
>> > presenting a small list of values in a drop-down menu. The current<br>
values<br>
>> > are what we derived for example data sets and discussion but they can<br>
>> > undoubtedly be extended and improved.<br>
>> ><br>
>> ><br>
>> ><br>
>> > We capture ?bucket? type measures through a combination of<br>
samplingEffort,<br>
>> > samplingGeometry and samplingUnit. For example, a pitfall trap (in a<br>
point<br>
>> > location) left out for 16 days might have samplingEffort: 16,<br>
>> > samplingGeometry: point and samplingUnit: day. Three m^2 quadrats in a<br>
>> > shore survey might have samplingEffort: 3, samplingGeometry: area and<br>
>> > samplingUnit: m^2.<br>
>> ><br>
>> ><br>
>> ><br>
>> > It would be very useful to see your compilation of scope, effort and<br>
>> > completeness measures to see if we can express them in our model and/or<br>
if<br>
>> > we need to reconsider our approach.<br>
>> ><br>
>> ><br>
>> ><br>
>> > ?amonn<br>
>> ><br>
>> ><br>
>> ><br>
>> > [1] <a href="http://eubon-ipt.gbif.org/resource.do?r=ionian-brackish-lagoon" target="_blank">http://eubon-ipt.gbif.org/resource.do?r=ionian-brackish-lagoon</a><br>
>> ><br>
>> ><br>
>> ><br>
>> > *From:* <a href="mailto:tdwg-content-bounces@lists.tdwg.org">tdwg-content-bounces@lists.tdwg.org</a> [mailto:<br>
>> > <a href="mailto:tdwg-content-bounces@lists.tdwg.org">tdwg-content-bounces@lists.tdwg.org</a>] *On Behalf Of *Markus D?ring<br>
>> > *Sent:* 20 August 2014 23:47<br>
>> > *To:* Robert Guralnick<br>
>> ><br>
>> > *Cc:* TDWG Content Mailing List<br>
>> > *Subject:* Re: [tdwg-content] Darwin Core: proposed news terms for<br>
>> > expressing sample data<br>
>> ><br>
>> ><br>
>> ><br>
>> > Rob,<br>
>> ><br>
>> ><br>
>> ><br>
>> > this proposal if for monitoring surveys really, not to be confused with<br>
>> > material samples like environmental or tissue samples which have a<br>
distinct<br>
>> > new dwc class MaterialSample.<br>
>> ><br>
>> ><br>
>> ><br>
>> > We tend to overload the term sampling a lot and it helps treating<br>
material<br>
>> > samples different from pure observational "sampling". That is why the<br>
>> > existing Event class was used as the core and classic Occurrence<br>
records as<br>
>> > extensions. A classic example is a vegetation survey where each plot<br>
>> > represents an Event record and each recorded species in that plot will<br>
be<br>
>> > an Occurrence extension record with a given quantity. Darwin Core<br>
already<br>
>> > offers individualCount to specify quantity, but it is a very specific<br>
way<br>
>> > of measuring "abundance" restricted to only some use cases. Abiotic<br>
>> > measurements about the plot (e.g. soil type, pH, temperature) can be<br>
>> > published using the measurements or facts extension linked to the Event<br>
>> > core.<br>
>> ><br>
>> ><br>
>> ><br>
>> > Markus<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > On 20 Aug 2014, at 20:08, Robert Guralnick<br>
<<a href="mailto:Robert.Guralnick@colorado.edu">Robert.Guralnick@colorado.edu</a>><br>
>> > wrote:<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > Anne -- I don't know the answers! These are questions for Eamonn. I<br>
>> > would presume that a sample could be a jumble of species or even just<br>
water<br>
>> > or soil samples, and biomass would refer to that sample - but maybe<br>
that<br>
>> > isn't a use case being considered? The examples given in the longer<br>
>> > document all link an event_id to species name and some measure of<br>
quantity<br>
>> > for that species (to the species, not an individual specimen), so I<br>
assume<br>
>> > that is the prevailing (or only) case?<br>
>> ><br>
>> > Best, Rob<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > On Wed, Aug 20, 2014 at 11:56 AM, Anne Thessen <<a href="mailto:annethessen@gmail.com">annethessen@gmail.com</a>><br>
>> > wrote:<br>
>> ><br>
>> > Hi Rob<br>
>> > I would like to respond to your item number 2.<br>
>> > From my perspective, I deal with lots of published descriptions of<br>
taxa.<br>
>> > The text might say something like "I saw species A in the Chesapeake<br>
Bay,<br>
>> > the Adriatic Sea and the Indian Ocean and the biomass is 5 - 9 grams".<br>
The<br>
>> > biomass range obviously corresponds to at least three different<br>
>> > occurrences, but how to divide the biomass data? I would love to be<br>
able to<br>
>> > have an *event* to attach it all to. There is almost two different<br>
levels<br>
>> > of events - a sampling event and a "study event". The "study event"<br>
would<br>
>> > correspond to the type of event I would like to use in the above<br>
example.<br>
>> > It may not be ideal, but for the old literature that might be the best<br>
we<br>
>> > can do.<br>
>> > I have to admit that I don't know enough about trawl data to understand<br>
>> > why an event core would be a problem. It seems that the trawl would be<br>
an<br>
>> > event and each biomass measure (of each fish) would be attached to a<br>
>> > separate occurrence which is attached to that event. Am I understanding<br>
>> > this wrong?<br>
>> > btw - I found a workaround for the example I gave, so it's not<br>
impossible<br>
>> > to model with the current structure....<br>
>> > Anne<br>
>> ><br>
>> ><br>
>> ><br>
>> > On 8/20/2014 1:16 PM, Robert Guralnick wrote:<br>
>> ><br>
>> ><br>
>> ><br>
>> > ?amonn et al. --- Thanks for the clarifications. I think these help a<br>
ton<br>
>> > but it raises a couple more questions for me.<br>
>> ><br>
>> ><br>
>> ><br>
>> > 1) I am surprised that you plan to use of MeasurementorFact extension<br>
in<br>
>> > relation to the Event core, which seems like a novel (or perhaps<br>
awkward or<br>
>> > unintended?) mechanism for capturing environmental data, but the same<br>
>> > extension was not be seen as relevant for describing samples? Can you<br>
>> > explain more about the thinking there?<br>
>> ><br>
>> ><br>
>> ><br>
>> > 2) There may be a subtle issue here extending "Event" to be more what<br>
you<br>
>> > call a "Sampling Event Core". My read of this is that Darwin Core<br>
serves<br>
>> > as a way to deal with point occurrences and Event reflects the context<br>
of a<br>
>> > single capture event (whether a single observation, or a bulk sample<br>
>> > capture). The changes recommended seem to dramatically extend and<br>
change<br>
>> > that meaning? Its simply a question that I don't have answer to, but<br>
is<br>
>> > Darwin Core, the right vehicle to start capturing repeated measures of<br>
>> > biomass values from trawls? I don't have answer but man, terms like<br>
>> > quantityType (as a property of occurrence?) give me pause.<br>
>> ><br>
>> ><br>
>> ><br>
>> > 3) Is Sampling Unit a controlled vocabulary? For another project, I<br>
have<br>
>> > looked through - and captured scope, effort and completeness measures<br>
from<br>
>> > - a large number of published biotic area inventories. The vast<br>
majorities<br>
>> > of these are measured in units like bucket hours, or trap nights. Is a<br>
>> > "bucket" part of SamplingGeometry or Sampling Unit? I'd be happy to<br>
send<br>
>> > along all the many examples of how biotic inventories of an area are<br>
>> > completed and perhaps it might be good to see how those might be<br>
>> > represented using the terms you are proposing?<br>
>> ><br>
>> ><br>
>> ><br>
>> > Best, Rob<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > On Wed, Aug 20, 2014 at 10:16 AM, Richard Pyle<br>
<<a href="mailto:deepreef@bishopmuseum.org">deepreef@bishopmuseum.org</a>><br>
>> > wrote:<br>
>> ><br>
>> > Same here ? Events are central to the work that we do.<br>
>> ><br>
>> ><br>
>> ><br>
>> > Aloha,<br>
>> ><br>
>> > Rich<br>
>> ><br>
>> ><br>
>> ><br>
>> > *From:* <a href="mailto:tdwg-content-bounces@lists.tdwg.org">tdwg-content-bounces@lists.tdwg.org</a> [mailto:<br>
>> > <a href="mailto:tdwg-content-bounces@lists.tdwg.org">tdwg-content-bounces@lists.tdwg.org</a>] *On Behalf Of *Anne Thessen<br>
>> > *Sent:* Wednesday, August 20, 2014 2:59 AM<br>
>> > *To:* <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>> ><br>
>> ><br>
>> > *Subject:* Re: [tdwg-content] Darwin Core: proposed news terms for<br>
>> > expressing sample data<br>
>> ><br>
>> ><br>
>> ><br>
>> > Hello<br>
>> > I would just like to comment on *event core*.<br>
>> > I've been doing a lot of work translating published data into Darwin<br>
Core.<br>
>> > During that process I've wished several times that I could use Event as<br>
>> > core. I am happy to hear about that proposed change. It will make it<br>
easier<br>
>> > to model the data I am working with.<br>
>> > Anne<br>
>> ><br>
>> > On 8/20/2014 7:04 AM, ?amonn ? Tuama [GBIF] wrote:<br>
>> ><br>
>> > Hi Rob,<br>
>> ><br>
>> ><br>
>> ><br>
>> > Thank you for the feedback. I have tried to address the two main issues<br>
>> > you raise below. At the outset, I would like to emphasise that much of<br>
this<br>
>> > work is taking place in the context of the EU BON project which<br>
includes a<br>
>> > task on developing/enhancing tools and standards for data sharing with<br>
a<br>
>> > particular focus on the IPT for publishing sample-based data. So, we<br>
were<br>
>> > constrained by the need to publish sample-based data sets in the Darwin<br>
>> > Core Archive format and to demonstrate practical application using a<br>
>> > working prototype. When the discussion on the TDWG list faded out, we<br>
took<br>
>> > it to our EU BON partners whose requirements were essential input to<br>
>> > further development. We recognise that these discussions took place<br>
away<br>
>> > from TDWG (although the TDWG/EU BON contributors overlapped) and this<br>
is<br>
>> > the reason we are presenting the outcomes here for further<br>
consideration.<br>
>> ><br>
>> ><br>
>> ><br>
>> > **Event core**<br>
>> ><br>
>> > As the SIGS report indicated, sample data can be modelled in Darwin<br>
Core<br>
>> > Archives using either Occurrence or Event as core. This was the<br>
starting<br>
>> > point for our evaluation but as things progressed the data wrangling<br>
pushed<br>
>> > the model back towards the Event core. We actually went through the<br>
>> > exercise of mapping multiple test datasets in an iterative process<br>
spanning<br>
>> > several months' work. In the end, we found that using an Event core<br>
better<br>
>> > matched the typical sample data we were dealing with, allowing use of a<br>
>> > measurement-or-fact extension to be included for the efficient<br>
expression<br>
>> > of environmental information associated with the event. The choice<br>
comes<br>
>> > down to an Occurrence core or an Event core + Occurrence extension. In<br>
both<br>
>> > cases, the true observation records are Occurrences. The big difference<br>
is<br>
>> > what type the core has and therefore to which kind of records you can<br>
>> > attach further facts and extra information with DwC-A extensions. Many<br>
>> > sampling datasets have very rich information about the site and event,<br>
so<br>
>> > it is very natural to hang facts from an Event core. When picking the<br>
>> > Occurrence core those facts would have to be repeated for each and<br>
every<br>
>> > occurrence record. Moreover, our approach doesn?t stop anyone from<br>
using<br>
>> > the Occurrence core if they so wish. This just provides a different<br>
option<br>
>> > for datasets that better fit an Event core model.<br>
>> ><br>
>> ><br>
>> ><br>
>> > I want to stress that we are not building a ?specific IPT version? to<br>
>> > support an Event core but, rather, we adapted the IPT so that it can be<br>
>> > configured to support any generic ?core + extension? format to enable<br>
its<br>
>> > use for exploration of more data formats. This is part of the core<br>
>> > codebase and there were no custom forks of the IPT for this work. Our<br>
view<br>
>> > at GBIF is that if there are significant numbers of data publishers who<br>
are<br>
>> > keen to adopt, promote and use a (any) format, and the tools can be<br>
>> > configured to do so, then we should support it, and, if necessary, use<br>
a<br>
>> > custom namespace.<br>
>> ><br>
>> ><br>
>> ><br>
>> > **New terms around abundance**<br>
>> ><br>
>> > Yes, the discussion on TDWG did fade out but it was clear that the term<br>
>> > ?abundance? as recommended by the SIGS report (along with<br>
>> > abundanceAsPercent) was confusing many when we were looking for term(s)<br>
>> > that reported quantitative measures of organisms in a sample. It also<br>
>> > became clear we would need to be able to state the type of quantity<br>
being<br>
>> > measured. An alternative suggestion for using the MeasurementsOrFact<br>
class<br>
>> > was immediately shot down.<br>
>> ><br>
>> > As some of our main use cases were coming from the EU BON project,<br>
>> > discussion shifted to that forum and consensus formed about the<br>
currently<br>
>> > proposed terms. It was within this group that the additional terms<br>
>> > (samplingGeometry, samplingUnit, eventSeriesID) were proposed and where<br>
we<br>
>> > began testing with sample data sets.<br>
>> ><br>
>> ><br>
>> ><br>
>> > Best regards,<br>
>> ><br>
>> > ?amonn<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > *From:* <a href="mailto:robgur@gmail.com">robgur@gmail.com</a> [mailto:<a href="mailto:robgur@gmail.com">robgur@gmail.com</a> <<a href="mailto:robgur@gmail.com">robgur@gmail.com</a>>]<br>
*On<br>
>> > Behalf Of *Robert Guralnick<br>
>> > *Sent:* 19 August 2014 16:56<br>
>> > *To:* ?amonn ? Tuama [GBIF]<br>
>> > *Cc:* TDWG Content Mailing List<br>
>> > *Subject:* Re: [tdwg-content] Darwin Core: proposed news terms for<br>
>> > expressing sample data<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > Hi ?amonn --- I am curious about the outcomes presented in the SIGS<br>
>> > paper, in particular, this portion of the paper:<br>
>> ><br>
>> ><br>
>> ><br>
>> > "Solutions without introducing an event core in Darwin Core Archives:<br>
>> > During the review of the solutions for the uses cases, it became<br>
apparent<br>
>> > that either model could be applied to every use case. The core and<br>
>> > extensions bore a complementary relationship and between them could<br>
express<br>
>> > all the required information. The core simply provided the central<br>
anchor<br>
>> > in the star schema from which to join the additional information.<br>
>> > Therefore, using the Occurrence core, well established in the GBIF<br>
network<br>
>> > through uptake of the IPT, seemed more appropriate than inventing<br>
>> > CollectingEvent as an additional core type."<br>
>> ><br>
>> ><br>
>> ><br>
>> > That SIGS paper has John Wieczorek and you both as authors,<br>
including<br>
>> > many luminaries across the biodiversity standards spectrum. Given the<br>
>> > above, its curious to see the EventCore come back again, along with a<br>
>> > specific IPT version to support it.<br>
>> ><br>
>> ><br>
>> ><br>
>> > So I see two issues, conflated, in this post you just made. One is<br>
>> > the need for an EventCore at all, and the nature of relating Event and<br>
>> > Occurrence/Material Sample. The second is the introduction of new<br>
terms,<br>
>> > which seemingly have arrived after debate on similar terms - but framed<br>
>> > around abundance - stalled a year ago. To my mind, these both require<br>
some<br>
>> > further discussion, because I don't (necessarily) see TDWG community<br>
>> > coherence around either issue?<br>
>> ><br>
>> ><br>
>> ><br>
>> > Best, Rob<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > On Tue, Aug 19, 2014 at 6:11 AM, ?amonn ? Tuama [GBIF]<br>
<<a href="mailto:eotuama@gbif.org">eotuama@gbif.org</a>><br>
>> > wrote:<br>
>> ><br>
>> > Dear All,<br>
>> ><br>
>> ><br>
>> ><br>
>> > GBIF is committed to exploring ways in which the IPT and Darwin Core<br>
>> > Archive format can be extended for publishing sample-based data sets.<br>
In<br>
>> > association with the EU BON project [1], a customised version of the<br>
IPT<br>
>> > [2] has been deployed to test this using a special type of Darwin Core<br>
>> > Archive in which the core is an ?Event? with associated taxon<br>
occurrences<br>
>> > in an ?Occurrence? extension.<br>
>> ><br>
>> ><br>
>> ><br>
>> > The Darwin Core vocabulary already provides a rich set of terms with<br>
many<br>
>> > relevant for describing sample-based data. Synthesising several sources<br>
of<br>
>> > input (GBIF organised workshop on sample data, May 2013 [3],<br>
discussions on<br>
>> > the TDWG mailing list in late 2013; internal discussion among EU BON<br>
>> > project partners), five new terms relating to sample data were<br>
identified<br>
>> > as essential. The complete model including these new terms are fully<br>
>> > described with examples in the online document ?Publishing sample data<br>
>> > using the GBIF IPT? [4].<br>
>> ><br>
>> ><br>
>> ><br>
>> > As a first step towards ratification, we would like to register the new<br>
>> > terms in the DwC Google Code tracker [5] if there are no major<br>
objections<br>
>> > on this list. The five terms are:<br>
>> ><br>
>> ><br>
>> ><br>
>> > 1. *quantity*: the number or enumeration value of the quantityType<br>
>> > (e.g., individuals, biomass, biovolume, BraunBlanquetScale) per<br>
>> > samplingUnit or a percentage measure recorded for the sample.<br>
>> ><br>
>> ><br>
>> ><br>
>> > 2. *quantityType*: : the entity being referred to by quantity,<br>
>> > e.g., individuals, biomass, %species, scale type.<br>
>> ><br>
>> ><br>
>> ><br>
>> > 3. *samplingGeometry*: an indication of what kind of space was<br>
>> > sampled; select from point, line, area or volume.<br>
>> ><br>
>> ><br>
>> ><br>
>> > 4. *samplingUnit*: the unit of measurement used for reporting the<br>
>> > quantity in the sample, e.g., minute, hour, day, metre, metre^2,<br>
metre^3.<br>
>> > It is combined with quantity and quantityType to provide the complete<br>
>> > measurement, e.g., 9 individuals per day, 4 biomass-gm per metre^2.<br>
>> ><br>
>> ><br>
>> ><br>
>> > 5. *eventSeriesID*: an identifier for a set of events that are<br>
>> > associated in some way, e.g., a monitoring series; may be a global<br>
unique<br>
>> > identifier or an identifier specific to the series.<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > Best regards,<br>
>> ><br>
>> ><br>
>> ><br>
>> > ?amonn<br>
>> ><br>
>> ><br>
>> ><br>
>> > [1] <a href="http://eubon.eu" target="_blank">http://eubon.eu</a><br>
>> ><br>
>> > [2] <a href="http://eubon-ipt.gbif.org" target="_blank">http://eubon-ipt.gbif.org</a><br>
>> ><br>
>> > [3]<br>
>> ><br>
<a href="http://www.standardsingenomics.org/index.php/sigen/article/view/sigs.4898640" target="_blank">http://www.standardsingenomics.org/index.php/sigen/article/view/sigs.4898640</a><br>
>> ><br>
>> > [4] <a href="http://links.gbif.org/sample_data_model" target="_blank">http://links.gbif.org/sample_data_model</a><br>
>> ><br>
>> > [5] <a href="https://code.google.com/p/darwincore/issues/list" target="_blank">https://code.google.com/p/darwincore/issues/list</a><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > ____________________________________________________<br>
>> ><br>
>> > *?amonn ? Tuama, M.Sc., Ph.D. (<a href="mailto:eotuama@gbif.org">eotuama@gbif.org</a> <<a href="mailto:eotuama@gbif.org">eotuama@gbif.org</a>>), *<br>
>> ><br>
>> > *Senior Programme Officer for Interoperability, *<br>
>> ><br>
>> > *Global Biodiversity Information Facility Secretariat, *<br>
>> ><br>
>> > *Universitetsparken 15, DK-2100, Copenhagen ?, DENMARK*<br>
>> ><br>
>> > *Phone: <a href="tel:%2B45%203532%201494" value="+4535321494">+45 3532 1494</a> <%2B45%203532%201494>; Fax: <a href="tel:%2B45%203532%201480" value="+4535321480">+45 3532 1480</a><br>
>> > <%2B45%203532%201480>*<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> > tdwg-content mailing list<br>
>> > <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>> > <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> ><br>
>> > tdwg-content mailing list<br>
>> ><br>
>> > <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>> ><br>
>> > <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>> ><br>
>> ><br>
>> ><br>
>> > --<br>
>> ><br>
>> > Anne E. Thessen, Ph.D.<br>
>> ><br>
>> > The Data Detektiv, Owner and Founder<br>
>> ><br>
>> > Ronin Institute, Research Scholar<br>
>> ><br>
>> > <a href="tel:443.225.9185" value="+14432259185">443.225.9185</a><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> > tdwg-content mailing list<br>
>> > <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>> > <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > --<br>
>> ><br>
>> > Anne E. Thessen, Ph.D.<br>
>> ><br>
>> > The Data Detektiv, Owner and Founder<br>
>> ><br>
>> > Ronin Institute, Research Scholar<br>
>> ><br>
>> > <a href="tel:443.225.9185" value="+14432259185">443.225.9185</a><br>
>> ><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> > tdwg-content mailing list<br>
>> > <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>> > <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>> ><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> > tdwg-content mailing list<br>
>> > <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>> > <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>> ><br>
>> ><br>
>> -------------- next part --------------<br>
>> An HTML attachment was scrubbed...<br>
>> URL:<br>
<a href="http://lists.tdwg.org/pipermail/tdwg-content/attachments/20140821/4b338606/a
ttachment-0001.html" target="_blank">http://lists.tdwg.org/pipermail/tdwg-content/attachments/20140821/4b338606/a<br>
ttachment-0001.html</a><br>
>><br>
>> ------------------------------<br>
>><br>
>> Message: 2<br>
>> Date: Fri, 22 Aug 2014 08:54:07 +0200<br>
>> From: "Donald Hobern [GBIF]" <<a href="mailto:dhobern@gbif.org">dhobern@gbif.org</a>><br>
>> Subject: Re: [tdwg-content] Darwin Core: proposed news terms for<br>
>> expressing sample data<br>
>> To: "'Matt Jones'" <<a href="mailto:jones@nceas.ucsb.edu">jones@nceas.ucsb.edu</a>>, '?amonn ? Tuama [GBIF]'<br>
>> <<a href="mailto:eotuama@gbif.org">eotuama@gbif.org</a>><br>
>> Cc: 'TDWG Content Mailing List' <<a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a>><br>
>> Message-ID: <003401cfbdd5$de52b640$9af822c0$@<a href="http://gbif.org" target="_blank">gbif.org</a>><br>
>> Content-Type: text/plain; charset="utf-8"<br>
>><br>
>> Hi Matt,<br>
>><br>
>><br>
>><br>
>> I?ll take the chance to make a few quick comments here, because I believe<br>
that this work is of massive importance.<br>
>><br>
>><br>
>><br>
>> Clearly DwC should avoid trying to duplicate well-standardised models and<br>
protocols. However at the same time, there is enormous value for producers<br>
and consumers of DwC to benefit from richer data on the events and methods<br>
associated with individual species occurrences. I have never seen DwC as<br>
purely an Occurrence exchange syntax. I see it (from GBIF?s standpoint)<br>
more closely as a mechanism for diverse parties to pool the evidence they<br>
have for the occurrence of any species including associated information<br>
and/or actionable links to associated information. Users coming from this<br>
perspective certainly need (and are demanding) access to all the evidence<br>
that can be mobilized to serve as supporting evidence and they also need the<br>
ability to understand the significance of these records. Abundance<br>
measures, levels of effort, use of consistent methods and redetection of<br>
individual organisms are all part of this. DwC should be able to transmit<br>
as much data as publishers cho<br>
>> ose to share on such aspects as part of their publishing of DwC. Users<br>
of DwC carrying out species modeling, threat assessment or community<br>
analyses will benefit from rapid ways to filter data for those which derive<br>
from standardized sampling events, to understand relative abundance within<br>
samples, etc. Many publishers of DwC are currently sharing stripped-down<br>
subsets of data and wish to give more information on these points. Users<br>
are certainly demanding it.<br>
>><br>
>><br>
>><br>
>> The challenge is finding the sweet spot, the achievable, non-destructive<br>
overlap between DwC and the proper domain of models better designed to<br>
handle the representation of complex systems outside DwC?s current domain.<br>
If this is done correctly, there should be paths that enable us to generate<br>
O&E (and maybe OBOE) compatible data from data that publishers only serve as<br>
augmented DwC.<br>
>><br>
>><br>
>><br>
>> I?ll also note that this has been a prominent area of discussion now for<br>
several years. Many of us believe strongly that this is one of the most<br>
important ways in which we need to close arbitrary gaps between data silos.<br>
It?s a prominent part of the GBIF work programme for 2014-2016.<br>
>><br>
>><br>
>><br>
>> Very best wishes,<br>
>><br>
>><br>
>><br>
>> Donald<br>
>><br>
>><br>
>><br>
>> From: <a href="mailto:tdwg-content-bounces@lists.tdwg.org">tdwg-content-bounces@lists.tdwg.org</a><br>
[mailto:<a href="mailto:tdwg-content-bounces@lists.tdwg.org">tdwg-content-bounces@lists.tdwg.org</a>] On Behalf Of Matt Jones<br>
>> Sent: Friday, August 22, 2014 4:52 AM<br>
>> To: ?amonn ? Tuama [GBIF]<br>
>> Cc: TDWG Content Mailing List<br>
>> Subject: Re: [tdwg-content] Darwin Core: proposed news terms for<br>
expressing sample data<br>
>><br>
>><br>
>><br>
>> This proposal is treading on ground that is quite similar to other<br>
observations and measurements standards for data exchange that are already<br>
mature, in particular:<br>
>><br>
>><br>
>><br>
>> * OGC Observations and Measurements<br>
(<a href="http://www.opengeospatial.org/standards/om" target="_blank">http://www.opengeospatial.org/standards/om</a>)<br>
>><br>
>> * Extensible Observation Ontology (OBOE;<br>
<a href="https://semtools.ecoinformatics.org/oboe" target="_blank">https://semtools.ecoinformatics.org/oboe</a>)<br>
>><br>
>><br>
>><br>
>> The former is a standard and broadly deployed, whereas the latter is part<br>
of a research program in the use of ontologies for measurements. Through<br>
collaboration between the two projects, they've been modified to be<br>
reasonably isomorphic, but O&M uses an XML serialization while OBOE uses an<br>
OWL-DL serialization. They largely express the same measurements and<br>
sampling model once one gets beyond the terminology differences.<br>
>><br>
>><br>
>><br>
>> So, I'm wondering if it make much sense to extend Darwin Core, which is<br>
at heart an Occurrence exchange syntax, into this measurements area that is<br>
well represented by these other existing specifications? I'm curious to<br>
hear why people would even want to do this. And if we do go down this path,<br>
won't we just end up with a new syntax that does essentially what O&M and<br>
OBOE do now?<br>
>><br>
>><br>
>><br>
>> Matt<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> On Thu, Aug 21, 2014 at 12:22 AM, ?amonn ? Tuama [GBIF]<br>
<<a href="mailto:eotuama@gbif.org">eotuama@gbif.org</a>> wrote:<br>
>><br>
>> Hi Rob, Anne, Rich,<br>
>><br>
>><br>
>><br>
>> I think Markus has answered your question as to why we opted for an Event<br>
core which is being used in the sense described by Anne and Rich. For any<br>
event, you can have a list of species in an Occurrence extension and for<br>
each species, you can include quantity and quantityType, e.g., biomass, etc.<br>
The proposed term eventSeriesID was intended for linking together related<br>
events, although it now looks like parentEventID might be a better, more<br>
flexible term. The measurementOrFact extension is a good fit for capturing<br>
environmental information relating to an event. See, e.g., the Gialova<br>
Lagoon brackish water invertebrate test data set [1] where a set of 18<br>
environmental variables, including temp, pH, Rdx, particulate organic<br>
matter, dissolved oxygen, salinity, chlorophyll-a were measured for each<br>
sampling station-sampling period combination. An example mapping is:<br>
>><br>
>><br>
>><br>
>> Id measurementType measurementValue<br>
measurementUnit measurementRemarks<br>
>><br>
>> IA Tmp (sed) 21.5<br>
degree C Tmp (sed): temperature at the bottom<br>
surface<br>
>><br>
>><br>
>><br>
>> *Controlled vocabularies*<br>
>><br>
>> Ideally, the values for samplingUnit and quantityType would be selected<br>
from controlled vocabularies. This is, effectively, what we do by presenting<br>
a small list of values in a drop-down menu. The current values are what we<br>
derived for example data sets and discussion but they can undoubtedly be<br>
extended and improved.<br>
>><br>
>><br>
>><br>
>> We capture ?bucket? type measures through a combination of<br>
samplingEffort, samplingGeometry and samplingUnit. For example, a pitfall<br>
trap (in a point location) left out for 16 days might have samplingEffort:<br>
16, samplingGeometry: point and samplingUnit: day. Three m^2 quadrats in a<br>
shore survey might have samplingEffort: 3, samplingGeometry: area and<br>
samplingUnit: m^2.<br>
>><br>
>><br>
>><br>
>> It would be very useful to see your compilation of scope, effort and<br>
completeness measures to see if we can express them in our model and/or if<br>
we need to reconsider our approach.<br>
>><br>
>><br>
>><br>
>> ?amonn<br>
>><br>
>><br>
>><br>
>> [1] <a href="http://eubon-ipt.gbif.org/resource.do?r=ionian-brackish-lagoon" target="_blank">http://eubon-ipt.gbif.org/resource.do?r=ionian-brackish-lagoon</a><br>
>><br>
>><br>
>><br>
>> From: <a href="mailto:tdwg-content-bounces@lists.tdwg.org">tdwg-content-bounces@lists.tdwg.org</a><br>
[mailto:<a href="mailto:tdwg-content-bounces@lists.tdwg.org">tdwg-content-bounces@lists.tdwg.org</a>] On Behalf Of Markus D?ring<br>
>> Sent: 20 August 2014 23:47<br>
>> To: Robert Guralnick<br>
>><br>
>><br>
>> Cc: TDWG Content Mailing List<br>
>> Subject: Re: [tdwg-content] Darwin Core: proposed news terms for<br>
expressing sample data<br>
>><br>
>><br>
>><br>
>> Rob,<br>
>><br>
>><br>
>><br>
>> this proposal if for monitoring surveys really, not to be confused with<br>
material samples like environmental or tissue samples which have a distinct<br>
new dwc class MaterialSample.<br>
>><br>
>><br>
>><br>
>> We tend to overload the term sampling a lot and it helps treating<br>
material samples different from pure observational "sampling". That is why<br>
the existing Event class was used as the core and classic Occurrence records<br>
as extensions. A classic example is a vegetation survey where each plot<br>
represents an Event record and each recorded species in that plot will be an<br>
Occurrence extension record with a given quantity. Darwin Core already<br>
offers individualCount to specify quantity, but it is a very specific way of<br>
measuring "abundance" restricted to only some use cases. Abiotic<br>
measurements about the plot (e.g. soil type, pH, temperature) can be<br>
published using the measurements or facts extension linked to the Event<br>
core.<br>
>><br>
>><br>
>><br>
>> Markus<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> On 20 Aug 2014, at 20:08, Robert Guralnick<br>
<<a href="mailto:Robert.Guralnick@colorado.edu">Robert.Guralnick@colorado.edu</a>> wrote:<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> Anne -- I don't know the answers! These are questions for Eamonn. I<br>
would presume that a sample could be a jumble of species or even just water<br>
or soil samples, and biomass would refer to that sample - but maybe that<br>
isn't a use case being considered? The examples given in the longer<br>
document all link an event_id to species name and some measure of quantity<br>
for that species (to the species, not an individual specimen), so I assume<br>
that is the prevailing (or only) case?<br>
>><br>
>> Best, Rob<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> On Wed, Aug 20, 2014 at 11:56 AM, Anne Thessen <<a href="mailto:annethessen@gmail.com">annethessen@gmail.com</a>><br>
wrote:<br>
>><br>
>> Hi Rob<br>
>> I would like to respond to your item number 2.<br>
>> >From my perspective, I deal with lots of published descriptions of taxa.<br>
The text might say something like "I saw species A in the Chesapeake Bay,<br>
the Adriatic Sea and the Indian Ocean and the biomass is 5 - 9 grams". The<br>
biomass range obviously corresponds to at least three different occurrences,<br>
but how to divide the biomass data? I would love to be able to have an<br>
*event* to attach it all to. There is almost two different levels of events<br>
- a sampling event and a "study event". The "study event" would correspond<br>
to the type of event I would like to use in the above example. It may not be<br>
ideal, but for the old literature that might be the best we can do.<br>
>> I have to admit that I don't know enough about trawl data to understand<br>
why an event core would be a problem. It seems that the trawl would be an<br>
event and each biomass measure (of each fish) would be attached to a<br>
separate occurrence which is attached to that event. Am I understanding this<br>
wrong?<br>
>> btw - I found a workaround for the example I gave, so it's not impossible<br>
to model with the current structure....<br>
>> Anne<br>
>><br>
>><br>
>><br>
>> On 8/20/2014 1:16 PM, Robert Guralnick wrote:<br>
>><br>
>><br>
>><br>
>> ?amonn et al. --- Thanks for the clarifications. I think these help a<br>
ton but it raises a couple more questions for me.<br>
>><br>
>><br>
>><br>
>> 1) I am surprised that you plan to use of MeasurementorFact extension<br>
in relation to the Event core, which seems like a novel (or perhaps awkward<br>
or unintended?) mechanism for capturing environmental data, but the same<br>
extension was not be seen as relevant for describing samples? Can you<br>
explain more about the thinking there?<br>
>><br>
>><br>
>><br>
>> 2) There may be a subtle issue here extending "Event" to be more what<br>
you call a "Sampling Event Core". My read of this is that Darwin Core<br>
serves as a way to deal with point occurrences and Event reflects the<br>
context of a single capture event (whether a single observation, or a bulk<br>
sample capture). The changes recommended seem to dramatically extend and<br>
change that meaning? Its simply a question that I don't have answer to, but<br>
is Darwin Core, the right vehicle to start capturing repeated measures of<br>
biomass values from trawls? I don't have answer but man, terms like<br>
quantityType (as a property of occurrence?) give me pause.<br>
>><br>
>><br>
>><br>
>> 3) Is Sampling Unit a controlled vocabulary? For another project, I have<br>
looked through - and captured scope, effort and completeness measures from -<br>
a large number of published biotic area inventories. The vast majorities of<br>
these are measured in units like bucket hours, or trap nights. Is a<br>
"bucket" part of SamplingGeometry or Sampling Unit? I'd be happy to send<br>
along all the many examples of how biotic inventories of an area are<br>
completed and perhaps it might be good to see how those might be represented<br>
using the terms you are proposing?<br>
>><br>
>><br>
>><br>
>> Best, Rob<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> On Wed, Aug 20, 2014 at 10:16 AM, Richard Pyle<br>
<<a href="mailto:deepreef@bishopmuseum.org">deepreef@bishopmuseum.org</a>> wrote:<br>
>><br>
>> Same here ? Events are central to the work that we do.<br>
>><br>
>><br>
>><br>
>> Aloha,<br>
>><br>
>> Rich<br>
>><br>
>><br>
>><br>
>> From: <a href="mailto:tdwg-content-bounces@lists.tdwg.org">tdwg-content-bounces@lists.tdwg.org</a><br>
[mailto:<a href="mailto:tdwg-content-bounces@lists.tdwg.org">tdwg-content-bounces@lists.tdwg.org</a>] On Behalf Of Anne Thessen<br>
>> Sent: Wednesday, August 20, 2014 2:59 AM<br>
>> To: <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>><br>
>><br>
>> Subject: Re: [tdwg-content] Darwin Core: proposed news terms for<br>
expressing sample data<br>
>><br>
>><br>
>><br>
>> Hello<br>
>> I would just like to comment on *event core*.<br>
>> I've been doing a lot of work translating published data into Darwin<br>
Core. During that process I've wished several times that I could use Event<br>
as core. I am happy to hear about that proposed change. It will make it<br>
easier to model the data I am working with.<br>
>> Anne<br>
>><br>
>> On 8/20/2014 7:04 AM, ?amonn ? Tuama [GBIF] wrote:<br>
>><br>
>> Hi Rob,<br>
>><br>
>><br>
>><br>
>> Thank you for the feedback. I have tried to address the two main issues<br>
you raise below. At the outset, I would like to emphasise that much of this<br>
work is taking place in the context of the EU BON project which includes a<br>
task on developing/enhancing tools and standards for data sharing with a<br>
particular focus on the IPT for publishing sample-based data. So, we were<br>
constrained by the need to publish sample-based data sets in the Darwin Core<br>
Archive format and to demonstrate practical application using a working<br>
prototype. When the discussion on the TDWG list faded out, we took it to our<br>
EU BON partners whose requirements were essential input to further<br>
development. We recognise that these discussions took place away from TDWG<br>
(although the TDWG/EU BON contributors overlapped) and this is the reason we<br>
are presenting the outcomes here for further consideration.<br>
>><br>
>><br>
>><br>
>> *Event core*<br>
>><br>
>> As the SIGS report indicated, sample data can be modelled in Darwin Core<br>
Archives using either Occurrence or Event as core. This was the starting<br>
point for our evaluation but as things progressed the data wrangling pushed<br>
the model back towards the Event core. We actually went through the exercise<br>
of mapping multiple test datasets in an iterative process spanning several<br>
months' work. In the end, we found that using an Event core better matched<br>
the typical sample data we were dealing with, allowing use of a<br>
measurement-or-fact extension to be included for the efficient expression of<br>
environmental information associated with the event. The choice comes down<br>
to an Occurrence core or an Event core + Occurrence extension. In both<br>
cases, the true observation records are Occurrences. The big difference is<br>
what type the core has and therefore to which kind of records you can attach<br>
further facts and extra information with DwC-A extensions. Many sampling<br>
datasets have very rich infor<br>
>> mation about the site and event, so it is very natural to hang facts<br>
from an Event core. When picking the Occurrence core those facts would have<br>
to be repeated for each and every occurrence record. Moreover, our approach<br>
doesn?t stop anyone from using the Occurrence core if they so wish. This<br>
just provides a different option for datasets that better fit an Event core<br>
model.<br>
>><br>
>><br>
>><br>
>> I want to stress that we are not building a ?specific IPT version? to<br>
support an Event core but, rather, we adapted the IPT so that it can be<br>
configured to support any generic ?core + extension? format to enable its<br>
use for exploration of more data formats. This is part of the core codebase<br>
and there were no custom forks of the IPT for this work. Our view at GBIF<br>
is that if there are significant numbers of data publishers who are keen to<br>
adopt, promote and use a (any) format, and the tools can be configured to do<br>
so, then we should support it, and, if necessary, use a custom namespace.<br>
>><br>
>><br>
>><br>
>> *New terms around abundance*<br>
>><br>
>> Yes, the discussion on TDWG did fade out but it was clear that the term<br>
?abundance? as recommended by the SIGS report (along with<br>
abundanceAsPercent) was confusing many when we were looking for term(s) that<br>
reported quantitative measures of organisms in a sample. It also became<br>
clear we would need to be able to state the type of quantity being measured.<br>
An alternative suggestion for using the MeasurementsOrFact class was<br>
immediately shot down.<br>
>><br>
>> As some of our main use cases were coming from the EU BON project,<br>
discussion shifted to that forum and consensus formed about the currently<br>
proposed terms. It was within this group that the additional terms<br>
(samplingGeometry, samplingUnit, eventSeriesID) were proposed and where we<br>
began testing with sample data sets.<br>
>><br>
>><br>
>><br>
>> Best regards,<br>
>><br>
>> ?amonn<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> From: <a href="mailto:robgur@gmail.com">robgur@gmail.com</a> [mailto:<a href="mailto:robgur@gmail.com">robgur@gmail.com</a>] On Behalf Of Robert<br>
Guralnick<br>
>> Sent: 19 August 2014 16:56<br>
>> To: ?amonn ? Tuama [GBIF]<br>
>> Cc: TDWG Content Mailing List<br>
>> Subject: Re: [tdwg-content] Darwin Core: proposed news terms for<br>
expressing sample data<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> Hi ?amonn --- I am curious about the outcomes presented in the SIGS<br>
paper, in particular, this portion of the paper:<br>
>><br>
>><br>
>><br>
>> "Solutions without introducing an event core in Darwin Core Archives:<br>
During the review of the solutions for the uses cases, it became apparent<br>
that either model could be applied to every use case. The core and<br>
extensions bore a complementary relationship and between them could express<br>
all the required information. The core simply provided the central anchor in<br>
the star schema from which to join the additional information. Therefore,<br>
using the Occurrence core, well established in the GBIF network through<br>
uptake of the IPT, seemed more appropriate than inventing CollectingEvent as<br>
an additional core type."<br>
>><br>
>><br>
>><br>
>> That SIGS paper has John Wieczorek and you both as authors, including<br>
many luminaries across the biodiversity standards spectrum. Given the<br>
above, its curious to see the EventCore come back again, along with a<br>
specific IPT version to support it.<br>
>><br>
>><br>
>><br>
>> So I see two issues, conflated, in this post you just made. One is<br>
the need for an EventCore at all, and the nature of relating Event and<br>
Occurrence/Material Sample. The second is the introduction of new terms,<br>
which seemingly have arrived after debate on similar terms - but framed<br>
around abundance - stalled a year ago. To my mind, these both require some<br>
further discussion, because I don't (necessarily) see TDWG community<br>
coherence around either issue?<br>
>><br>
>><br>
>><br>
>> Best, Rob<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> On Tue, Aug 19, 2014 at 6:11 AM, ?amonn ? Tuama [GBIF] <<a href="mailto:eotuama@gbif.org">eotuama@gbif.org</a>><br>
wrote:<br>
>><br>
>> Dear All,<br>
>><br>
>><br>
>><br>
>> GBIF is committed to exploring ways in which the IPT and Darwin Core<br>
Archive format can be extended for publishing sample-based data sets. In<br>
association with the EU BON project [1], a customised version of the IPT [2]<br>
has been deployed to test this using a special type of Darwin Core Archive<br>
in which the core is an ?Event? with associated taxon occurrences in an<br>
?Occurrence? extension.<br>
>><br>
>><br>
>><br>
>> The Darwin Core vocabulary already provides a rich set of terms with many<br>
relevant for describing sample-based data. Synthesising several sources of<br>
input (GBIF organised workshop on sample data, May 2013 [3], discussions on<br>
the TDWG mailing list in late 2013; internal discussion among EU BON project<br>
partners), five new terms relating to sample data were identified as<br>
essential. The complete model including these new terms are fully described<br>
with examples in the online document ?Publishing sample data using the GBIF<br>
IPT? [4].<br>
>><br>
>><br>
>><br>
>> As a first step towards ratification, we would like to register the new<br>
terms in the DwC Google Code tracker [5] if there are no major objections on<br>
this list. The five terms are:<br>
>><br>
>><br>
>><br>
>> 1. quantity: the number or enumeration value of the quantityType<br>
(e.g., individuals, biomass, biovolume, BraunBlanquetScale) per samplingUnit<br>
or a percentage measure recorded for the sample.<br>
>><br>
>><br>
>><br>
>> 2. quantityType: : the entity being referred to by quantity, e.g.,<br>
individuals, biomass, %species, scale type.<br>
>><br>
>><br>
>><br>
>> 3. samplingGeometry: an indication of what kind of space was<br>
sampled; select from point, line, area or volume.<br>
>><br>
>><br>
>><br>
>> 4. samplingUnit: the unit of measurement used for reporting the<br>
quantity in the sample, e.g., minute, hour, day, metre, metre^2, metre^3.<br>
It is combined with quantity and quantityType to provide the complete<br>
measurement, e.g., 9 individuals per day, 4 biomass-gm per metre^2.<br>
>><br>
>><br>
>><br>
>> 5. eventSeriesID: an identifier for a set of events that are<br>
associated in some way, e.g., a monitoring series; may be a global unique<br>
identifier or an identifier specific to the series.<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> Best regards,<br>
>><br>
>><br>
>><br>
>> ?amonn<br>
>><br>
>><br>
>><br>
>> [1] <a href="http://eubon.eu" target="_blank">http://eubon.eu</a> <<a href="http://eubon.eu/" target="_blank">http://eubon.eu/</a>><br>
>><br>
>> [2] <a href="http://eubon-ipt.gbif.org" target="_blank">http://eubon-ipt.gbif.org</a> <<a href="http://eubon-ipt.gbif.org/" target="_blank">http://eubon-ipt.gbif.org/</a>><br>
>><br>
>> [3]<br>
<a href="http://www.standardsingenomics.org/index.php/sigen/article/view/sigs.4898640" target="_blank">http://www.standardsingenomics.org/index.php/sigen/article/view/sigs.4898640</a><br>
>><br>
>> [4] <<a href="http://links.gbif.org/sample_data_model" target="_blank">http://links.gbif.org/sample_data_model</a>><br>
<a href="http://links.gbif.org/sample_data_model" target="_blank">http://links.gbif.org/sample_data_model</a><br>
>><br>
>> [5] <a href="https://code.google.com/p/darwincore/issues/list" target="_blank">https://code.google.com/p/darwincore/issues/list</a><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> ____________________________________________________<br>
>><br>
>> ?amonn ? Tuama, M.Sc., Ph.D. (<a href="mailto:eotuama@gbif.org">eotuama@gbif.org</a>),<br>
>><br>
>> Senior Programme Officer for Interoperability,<br>
>><br>
>> Global Biodiversity Information Facility Secretariat,<br>
>><br>
>> Universitetsparken 15, DK-2100, Copenhagen ?, DENMARK<br>
>><br>
>> Phone: +45 3532 1494 <tel:%2B45%203532%201494> ; Fax: +45 3532 1480<br>
<tel:%2B45%203532%201480><br>
>><br>
>><br>
>><br>
>><br>
>> _______________________________________________<br>
>> tdwg-content mailing list<br>
>> <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>> <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> _______________________________________________<br>
>> tdwg-content mailing list<br>
>> <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>><br>
>> <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>><br>
>><br>
>><br>
>> --<br>
>> Anne E. Thessen, Ph.D.<br>
>> The Data Detektiv, Owner and Founder<br>
>> Ronin Institute, Research Scholar<br>
>> 443.225.9185<br>
>><br>
>><br>
>> _______________________________________________<br>
>> tdwg-content mailing list<br>
>> <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>> <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> --<br>
>> Anne E. Thessen, Ph.D.<br>
>> The Data Detektiv, Owner and Founder<br>
>> Ronin Institute, Research Scholar<br>
>> 443.225.9185<br>
>><br>
>><br>
>><br>
>> _______________________________________________<br>
>> tdwg-content mailing list<br>
>> <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>> <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>><br>
>><br>
>><br>
>><br>
>> _______________________________________________<br>
>> tdwg-content mailing list<br>
>> <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>> <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>><br>
>><br>
>><br>
>> -------------- next part --------------<br>
>> An HTML attachment was scrubbed...<br>
>> URL:<br>
<a href="http://lists.tdwg.org/pipermail/tdwg-content/attachments/20140822/a46f067d/a
ttachment-0001.html" target="_blank">http://lists.tdwg.org/pipermail/tdwg-content/attachments/20140822/a46f067d/a<br>
ttachment-0001.html</a><br>
>><br>
>> ------------------------------<br>
>><br>
>> _______________________________________________<br>
>> tdwg-content mailing list<br>
>> <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>> <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>><br>
>><br>
>> End of tdwg-content Digest, Vol 63, Issue 6<br>
>> *******************************************<br>
>><br>
>> _______________________________________________<br>
>> tdwg-content mailing list<br>
>> <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
>> <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> tdwg-content mailing list<br>
> <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
> <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> tdwg-content mailing list<br>
> <a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
> <a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
<br>
<br>
<br>
_______________________________________________<br>
tdwg-content mailing list<br>
<a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
<a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
<br>
<br>
_______________________________________________<br>
tdwg-content mailing list<br>
<a href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
<a href="http://lists.tdwg.org/mailman/listinfo/tdwg-content" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>John Deck<br>(541) 321-0689<br>
</div></div>