<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
A couple of points of clarification on what Chuck said:<br>
<br>
1. Audubon Core is more than an extension of Darwin Core - it's a
vocabulary that is a ratified TDWG technical specification standard
(despite what [1] says - nobody has managed to change its status from
"Draft" to "Current" on the website). So it stands on the same level
as Darwin Core even though it's terms describe different kinds of
things than DwC. All ratified TDWG standards have gone through peer
review, public comment, and approval by the TDWG Executive. That is
not the case with Plinian Core or GBIF extensions. <br>
<br>
2. The Darwin Core text guide that Chuck mentions IS actually part of
the Darwin Core Standard. TDWG Standards are composed of normative
("Type 1") documents that define the standard and non-normative ("Type
2") documents that "explain and justify" the standard. For Darwin
Core, there is a single normative RDF document that defines the
standard. It can be viewed at
<a class="moz-txt-link-freetext" href="https://code.google.com/p/darwincore/source/browse/trunk/rdf/dwctermshistory.rdf">https://code.google.com/p/darwincore/source/browse/trunk/rdf/dwctermshistory.rdf</a>
. There are 67 non-normative documents, which include the entire
Darwin Core website (including the text guide page). How do I know
that these documents comprise the standard? Because they are what I
get when I download and unzip the standard from
<a class="moz-txt-link-freetext" href="http://www.tdwg.org/standards/450/">http://www.tdwg.org/standards/450/</a> . It is one of TDWG's deficiencies
that it is not a simple matter to know what actually comprises a
standard and to view clearly-identified components of a standard
without having to download a zip archive. <br>
<br>
I'm not sure that I can articulate the difference between the Darwin
Core Text Guide component of the standard and a "Darwin Core Archive".
It might be correct to say that Darwin Core Archives are sets of files
that are compliant with the Darwin Core Text Guide. <br>
<br>
Steve<br>
<br>
[1] <a class="moz-txt-link-freetext" href="http://www.tdwg.org/standards/">http://www.tdwg.org/standards/</a><br>
<br>
Chuck Miller wrote:
<blockquote
cite="mid:059599138B8E5E4F9F90273125749230809BD29A@MBGMail02.mobot.org"
type="cite">
<meta http-equiv="Content-Type" content="text/html; ">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.hoenzb
{mso-style-name:hoenzb;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);">Rob,<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);">GBIF
is hard at work solving a problem for them. It seems natural for them
to do what works best for them, particularly dealing with the
volume/scale they do. And, Markus, Tim, Eamonn and others have been
deeply involved in Darwin Core standards implementation, inventing the
DwCA and the IPT in the first place. But, in using DwCA to get actual
work done for GBIF they have added a term here and there via their own
extensions. See <a moz-do-not-send="true"
href="http://tools.gbif.org/dwca-validator/extensions.do">http://tools.gbif.org/dwca-validator/extensions.do</a>.
It’s a long list of Registered and Stable extensions. What about
Audubon and Plinian Core? Aren’t they actually “extensions” to Darwin
Core, particularly in the “universal biodiversity data “ sense Darwin
Core seems to be evolving to? Maybe we need a “SampleCore” or
“EventCore” that organizes these sampling/event-related terms?<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);">GBIF
is creating a new “Event core” for DwCA to do what they need to do.
But, I don’t understand the problem with this. Occurrence core, Taxon
core, Event core – so what? The DwCA model allows anything to be the
core (ignoring IPT) and it does not actually restrict the terms used to
DwC only, certainly it already includes Dublin Core and EML. In fact,
lots of DwCA exchanges are being used outside GBIF now with extended
terms from Audubon Core, Plinian Core, and others. How is this bad?
It shows how flexible the DwCA data exchange structure is. And, it is
an exchange, not a data storage format. What’s the problem with an
Event core?<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);">As
an aside, I have said many times that the most confusing thing to our
community about DwCA is that it’s called <u>Darwin Core</u> Archive
when the structure is in fact not limited to only Darwin Core terms.
Any validatable XML term can be used – like </span><span
style="font-size: 10pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);"><a
moz-do-not-send="true"
href="http://rs.gbif.org/terms/1.0/verbatimLabel">http://rs.gbif.org/terms/1.0/verbatimLabel</a>
which is used in the How-To Guide Whales example. </span><span
style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);"> And
I commonly hear people at conferences and meetings now referring to
their plans to adopt “Darwin Core” and upon questioning discover they
are talking about the DwCA format, not really knowing the details of
it. So, the words “Darwin Core” have become ambiguous.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);">And
I want to note that Darwin Core Archive is not a TDWG standard. It is
a guideline for implementing the Darwin Core standard in text files. <a
moz-do-not-send="true" href="http://rs.tdwg.org/dwc/terms/guides/text/">http://rs.tdwg.org/dwc/terms/guides/text/</a>.
This is another point of confusion. GBIF describes DwCA as “an
internationally recognized biodiversity informatics data standard” <a
moz-do-not-send="true" href="http://www.gbif.org/resources/2551">http://www.gbif.org/resources/2551</a>,
but it’s not a TDWG standard, just a guideline. I think DwCA deserves
to be made a TDWG data exchange standard on its own, separate from
Darwin Core, and perhaps expanded in scope to embrace the other “Cores”
and relabeled to “Biodiversity Data Archive” or something.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);">Chuck<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size: 11pt; font-family: "Calibri","sans-serif"; color: rgb(31, 73, 125);"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span
style="font-size: 10pt; font-family: "Tahoma","sans-serif";">From:</span></b><span
style="font-size: 10pt; font-family: "Tahoma","sans-serif";">
<a class="moz-txt-link-abbreviated" href="mailto:tdwg-content-bounces@lists.tdwg.org">tdwg-content-bounces@lists.tdwg.org</a>
[<a class="moz-txt-link-freetext" href="mailto:tdwg-content-bounces@lists.tdwg.org">mailto:tdwg-content-bounces@lists.tdwg.org</a>] <b>On Behalf Of </b>Robert
Guralnick<br>
<b>Sent:</b> Thursday, August 28, 2014 10:58 AM<br>
<b>To:</b> John Deck<br>
<b>Cc:</b> TDWG Content Mailing List; Ramona Walls<br>
<b>Subject:</b> Re: [tdwg-content] tdwg-content Digest, Vol 63, Issue
6<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"> Hi all --- Ok, I think the scope of the issue
is quite clear. Let me summarize: 1) As Éamonn and the rest of GBIF
has made quite clear, "<span
style="font-size: 9.5pt; font-family: "Arial","sans-serif";">GBIF is
faced with the immediate task of making sample-based data discoverable
and accessible using its current ecosystem of tools" given a funding
mandate from EU-BON. 2) The solution for this problem is to develop
an Event-core and to promote new terms to the Darwin Core to make this
happen. I will note a small inconsistency here: the current
ecosystem standards and tools of is Darwin Core (as it stands) and
publishing systems such as IPT. That ecosystem of tools includes
mechanisms to extend Darwin Core where needed, via extensions. The <i>current</i>
ecosystem of tools doesn't include new Cores or new DwC terms, does it?</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family: "Arial","sans-serif";">
So this leads in nicely to the contentious issue(s) and places where
there seems to be discussions --- these have to do with the nature of
the changes suggested and the scope of those changes, both in terms of
an Event core and DwC term additions. Leaving aside the Event-core for
now, the key questions simply about term additions to the Darwin Core
that seem to be at heart here are: 1) Is the intent of the Darwin Core
to model surveys, which usually involve multiple kinds and types of
sampling over multiple sites using multiple methods? 2) Is the
solution to invent new terms for the Darwin Core if there are already
terms from other efforts, wouldn't we work with those existing efforts
to assure interoperability? </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family: "Arial","sans-serif";">
I appreciate the efforts of GBIF here fully, and am personally torn
because on the one hand, I fully agree with the goal of extending
Darwin Core to better represent richer biodiversity data. On the other
hand, I worry about process here and how to make that happen in a way
that isn't too hasty or locks us into just the opposite of what I think
many of us want with regards to sharing data more broadly than within
just one ecosystem of tools. </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family: "Arial","sans-serif";">Best,
Rob</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size: 9.5pt; font-family: "Arial","sans-serif";"> </span><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom: 12pt;"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Thu, Aug 28, 2014 at 6:30 AM, John Deck <<a
moz-do-not-send="true" href="mailto:jdeck@berkeley.edu" target="_blank">jdeck@berkeley.edu</a>>
wrote:<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"><span
style="font-family: "Arial","sans-serif"; color: rgb(80, 0, 80);">I
see the rational for enabling this in Darwin Core Archives and adding
the new terms. However, back to what Matt Jones brought up: "</span><span
style="font-size: 9.5pt; font-family: "Arial","sans-serif"; color: rgb(80, 0, 80);">won't
we just end up with a new syntax that does essentially what O&M and
OBOE do now?". </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom: 12pt;"><span
style="font-size: 9.5pt; font-family: "Arial","sans-serif"; color: rgb(80, 0, 80);">We
should include explicit references to existing terms/definitions that
encapsulate what we're talking about, e.g. in our MaterialSample
proposal last year we linked the an existing term in OBI, which has a
much richer description and context for MaterialSample than what we
considered (</span><span
style="font-family: "Arial","sans-serif"; color: rgb(80, 0, 80);"><a
moz-do-not-send="true"
href="https://code.google.com/p/darwincore/issues/detail?id=167"
target="_blank">https://code.google.com/p/darwincore/issues/detail?id=167</a>) </span><o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">Have we explored the possibility of doing this
with OBOE? I'm not suggesting we adopt OBOE wholesale, but it seems
like we have a good opportunity to enable better semantic linking with
that efforts.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">John<o:p></o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">On Thu, Aug 28, 2014 at 4:23 AM, Éamonn Ó Tuama
[GBIF] <<a moz-do-not-send="true" href="mailto:eotuama@gbif.org"
target="_blank">eotuama@gbif.org</a>> wrote:<o:p></o:p></p>
<p class="MsoNormal">Thanks, Ramona and Rob.<br>
<br>
I'd like to add a few points following on Markus's reply.<br>
<br>
I think your pressing of the need for a robust semantic model for<br>
biodiversity sample/survey data is incontestable – we do need one and it<br>
should enable rich data integration once it is defined and the tools and<br>
data standards to support it become available. However, GBIF is faced
with<br>
the immediate task of making sample-based data discoverable and
accessible<br>
using its current ecosystem of tools (IPT) and exchange standards (DwC;<br>
EML). Waiting for a functional, implementable semantic model and the
tools<br>
and support services for it is just not an option for us right now.<br>
<br>
We have already spend considerable time in analysing the merits of<br>
Occurrence core vs Event core and have opted for an Event core for
reasons<br>
previously given. I don’t believe we are trying to reconfigure Event
(“an<br>
action that occurs at a place and during a period of time”) and
regardless<br>
of whether we use Occurrence or Event, the need for some additional
terms<br>
arise (e.g., quantity, quantityType, samplingGeometry, samplingUnit).
Once<br>
the BCO model is available for uptake, it should be possible to develop
a<br>
mapping between it and the simple DwC sample model.<br>
<br>
So GBIF’s stance is that we need to take a two-pronged approach by
exploring<br>
how the IPT and DwC-A can be adapted for publishing sample-based data
in the<br>
near term while supporting the work of TDWG and groups such as the BCO
in<br>
advancing biodiversity informatics. GBIF has already engaged in the
work of<br>
the BCO and will continue to do so.<br>
<span style="color: rgb(136, 136, 136);"><br>
Éamonn</span><o:p></o:p></p>
<div>
<p class="MsoNormal"><br>
-----Original Message-----<br>
From: <a moz-do-not-send="true"
href="mailto:tdwg-content-bounces@lists.tdwg.org" target="_blank">tdwg-content-bounces@lists.tdwg.org</a><br>
[mailto:<a moz-do-not-send="true"
href="mailto:tdwg-content-bounces@lists.tdwg.org" target="_blank">tdwg-content-bounces@lists.tdwg.org</a>]
On Behalf Of Markus Döring<br>
Sent: 28 August 2014 12:44<br>
To: Ramona Walls<br>
Cc: TDWG Content Mailing List<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">Subject: Re: [tdwg-content] tdwg-content Digest,
Vol 63, Issue 6<br>
<br>
Hi Ramona & Rob,<br>
<br>
The Event proposal does not try to change the semantics of an Event, it
just<br>
uses the existing Darwin Core Event "class" at the core in Darwin Core<br>
archives. The actual change proposed is simply adding 3 new terms to the<br>
Event "group" to better share information about sampling methods &
efforts,<br>
extending the existing limited capabilities of Darwin Core which
already has<br>
the terms dwc:samplingProtocol and dwc:samplingEffort. It also proposes
2<br>
new terms for dealing with quantity of Occurrences, something that has
been<br>
discussed since 2012 now, when I had proposed a new abundance term [2].<br>
<br>
<br>
In general application of Darwin Core is not at all limited to
specimens and<br>
observations. It is used for sharing taxonomic datasets already and it's<br>
definition and goal is broad. Let me cite some of the introduction to
Darwin<br>
Core [1]:<br>
<br>
What is the Darwin Core?<br>
The Darwin Core is body of standards. It includes a glossary of terms
(in<br>
other contexts these might be called properties, elements, fields,
columns,<br>
attributes, or concepts) intended to facilitate the sharing of
information<br>
about biological diversity by providing reference definitions,
examples, and<br>
commentaries. The Darwin Core is primarily based on taxa, their
occurrence<br>
in nature as documented by observations, specimens, samples, and related<br>
information.<br>
<br>
Motivation: The Darwin Core standard was originally conceived to
facilitate<br>
the discovery, retrieval, and integration of information about modern<br>
biological specimens, their spatiotemporal occurrence, and their
supporting<br>
evidence housed in collections (physical or digital). The Darwin Core
today<br>
is broader in scope and more versatile. It is meant to provide a stable<br>
standard reference for sharing information on biological diversity. As a<br>
glossary of terms, the Darwin Core is meant to provide stable semantic<br>
definitions with the goal of being maximally reusable in a variety of<br>
contexts.<br>
<br>
<br>
Markus<br>
<br>
<br>
[1] <a moz-do-not-send="true" href="http://rs.tdwg.org/dwc/index.htm"
target="_blank">http://rs.tdwg.org/dwc/index.htm</a><br>
[2] <a moz-do-not-send="true"
href="https://code.google.com/p/darwincore/issues/detail?id=142"
target="_blank">https://code.google.com/p/darwincore/issues/detail?id=142</a><br>
<br>
<br>
--<br>
Markus Döring<br>
Software Developer<br>
Global Biodiversity Information Facility (GBIF)<br>
<a moz-do-not-send="true" href="mailto:mdoering@gbif.org"
target="_blank">mdoering@gbif.org</a><br>
<a moz-do-not-send="true" href="http://www.gbif.org" target="_blank">http://www.gbif.org</a><br>
<br>
<br>
<br>
<br>
<br>
On 27 Aug 2014, at 18:57, Ramona Walls <<a moz-do-not-send="true"
href="mailto:rlwalls2008@gmail.com" target="_blank">rlwalls2008@gmail.com</a>>
wrote:<br>
<br>
> I think it is important to consider the purpose of both Darwin
Core and<br>
DwC archives in deciding whether or not to expand them, but we should
use<br>
that consideration to address the question at hand, which is whether or
not<br>
to add an Event core and additional properties to describe events.<br>
><br>
> Describing the exchange format before the semantics is the wrong
way to<br>
go, given that we now have a framework for developing semantics.
Expanding<br>
Darwin Core before we adequately model survey data is bound to lead to<br>
problems later, when we try to retro-fit the semantics to Darwin Core
Event<br>
archives. This is exactly the problem we are running into now with
Occurance<br>
archives, and we have the opportunity to avoid it.<br>
><br>
> I suggest we first use existing ontologies to model survey data,
then deal<br>
with if and how to exchange that information in DwC-A. This is what I
was<br>
hinting at in my first email, but should have said more explicitly.<br>
><br>
> Ramona<br>
><br>
> ------------------------------------------------------<br>
> Ramona L. Walls, Ph.D.<br>
> Scientific Analyst, The iPlant Collaborative, University of Arizona<br>
> Research Associate, Bio5 Institute, University of Arizona<br>
> Laboratory Research Associate, New York Botanical Garden<br>
><br>
><br>
> On Wed, Aug 27, 2014 at 9:14 AM, Robert Guralnick<br>
<<a moz-do-not-send="true"
href="mailto:Robert.Guralnick@colorado.edu" target="_blank">Robert.Guralnick@colorado.edu</a>>
wrote:<br>
><br>
> It may be a sensible view for Darwin Core Archives and their
intended<br>
use, but Tim's email suggests we should be putting the method of
delivery<br>
ahead of the standard that delivers that content. If this was just
about<br>
DwC-As, why not develop a survey extension that links each occurrence to<br>
information about the survey process using the existing star-schema
methods<br>
we have in place? Why are we discussing adding terms to the Darwin
Core or<br>
trying to fully reconfigure what we call an Event? That is what is on
the<br>
table, not DwC-As and how we use them. Or am I missing something?<br>
><br>
> Best, Rob<br>
><br>
><br>
><br>
> On Wed, Aug 27, 2014 at 10:01 AM, Ramona Walls <<a
moz-do-not-send="true" href="mailto:rlwalls2008@gmail.com"
target="_blank">rlwalls2008@gmail.com</a>><br>
wrote:<br>
> Thanks, Tim, and yes, DwC-A as a view (but not necessarily the
primary<br>
archive) of data seems like the right point of view.<br>
><br>
> Ramona<br>
><br>
> ------------------------------------------------------<br>
> Ramona L. Walls, Ph.D.<br>
> Scientific Analyst, The iPlant Collaborative, University of Arizona<br>
> Research Associate, Bio5 Institute, University of Arizona<br>
> Laboratory Research Associate, New York Botanical Garden<br>
><br>
><br>
> On Wed, Aug 27, 2014 at 1:58 AM, Tim Robertson <<a
moz-do-not-send="true" href="mailto:trobertson@gbif.org"
target="_blank">trobertson@gbif.org</a>><br>
wrote:<br>
> Hi Ramona,<br>
><br>
> Those are good points, and I’d like to come back to the original
thinking<br>
behind the DwC-A.<br>
><br>
> It was designed and intended to be a simple way of exposing a
complete<br>
view of a dataset, primarily for building sophisticated indexes,
inventories<br>
and allowing basic analytics (e.g. GBIF.org being one sophisticated
index).<br>
We found that the star schema provided the flexibility to do a lot, and
with<br>
the bundled metadata (e.g. EML) was enough to trace provenance and allow<br>
users to determine if the dataset might be fit for various uses. In
many<br>
cases this represents the complete (e.g. lossless) view of a dataset.<br>
><br>
> What we are discussing here are far richer datasets, where
shoe-horning<br>
content into the star schema becomes lossy for some, although we’re
finding<br>
other cases where it is indeed lossless. I believe we should be
looking to<br>
harmonise ontologies / models etc as you mention but in parallel we
should<br>
define one or more star schema views that can still be used for
discovery /<br>
reporting / basic analytical purpose, and not long term archival of the<br>
dataset. The dataset would then have the canonical rich form and an<br>
additional DwC-A view. What I write here is applicable to all content
types<br>
of course.<br>
><br>
> Please also note that many people put supplementary files in the
DwC-A<br>
which are ignored by DwC-A readers but could be a way of keeping the
richer<br>
view in the bundle. If one wished you can describe those supplementary<br>
files in the EML document.<br>
><br>
> Does this gel with the view of others as well?<br>
><br>
> Cheers,<br>
> Tim<br>
><br>
><br>
><br>
> On 27 Aug 2014, at 02:55, Ramona Walls <<a
moz-do-not-send="true" href="mailto:rlwalls2008@gmail.com"
target="_blank">rlwalls2008@gmail.com</a>> wrote:<br>
><br>
>> I think Matt hit the nail on the head. Although Darwin Core
can be used<br>
to exchange survey data, it lacks the semantics and structure necessary
to<br>
archive the data without loss of information. I think the biodiversity<br>
community would be better served devoting energy to harmonizing existing<br>
technologies such as OGC, OBOE, and BCO, not to mention the many
database<br>
for storing plot or survey data. The goal should be to preserve the
data in<br>
the most informative manner possible.<br>
>><br>
>> There is a strong a case for wanting to search across all
evidence for<br>
occurences, including surveys and point occurences, so I can see
possible<br>
demand for a tool that would extract occurences from survey data to a
DwC<br>
archive. However, I am very concerned that making a DwC archive the
primary<br>
exchange format for survey or plot data commits us to a path of losing<br>
information from the start, for all but the simplest sampling schemas.<br>
>><br>
>> Ramona<br>
>><br>
>> ------------------------------------------------------<br>
>> Ramona L. Walls, Ph.D.<br>
>> Scientific Analyst, The iPlant Collaborative, University of
Arizona<br>
>> Research Associate, Bio5 Institute, University of Arizona<br>
>> Laboratory Research Associate, New York Botanical Garden<br>
>><br>
>><br>
>> On Fri, Aug 22, 2014 at 3:00 AM, <<a moz-do-not-send="true"
href="mailto:tdwg-content-request@lists.tdwg.org" target="_blank">tdwg-content-request@lists.tdwg.org</a>><br>
wrote:<br>
>> Send tdwg-content mailing list submissions to<br>
>> <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>><br>
>> To subscribe or unsubscribe via the World Wide Web, visit<br>
>> <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>> or, via email, send a message with subject or body 'help' to<br>
>> <a moz-do-not-send="true"
href="mailto:tdwg-content-request@lists.tdwg.org" target="_blank">tdwg-content-request@lists.tdwg.org</a><br>
>><br>
>> You can reach the person managing the list at<br>
>> <a moz-do-not-send="true"
href="mailto:tdwg-content-owner@lists.tdwg.org" target="_blank">tdwg-content-owner@lists.tdwg.org</a><br>
>><br>
>> When replying, please edit your Subject line so it is more
specific<br>
>> than "Re: Contents of tdwg-content digest..."<br>
>><br>
>><br>
>> Today's Topics:<br>
>><br>
>> 1. Re: Darwin Core: proposed news terms for expressing
sample<br>
>> data (Matt Jones)<br>
>> 2. Re: Darwin Core: proposed news terms for expressing
sample<br>
>> data (Donald Hobern [GBIF])<br>
>><br>
>><br>
>>
----------------------------------------------------------------------<br>
>><br>
>> Message: 1<br>
>> Date: Thu, 21 Aug 2014 18:52:06 -0800<br>
>> From: Matt Jones <<a moz-do-not-send="true"
href="mailto:jones@nceas.ucsb.edu" target="_blank">jones@nceas.ucsb.edu</a>><br>
>> Subject: Re: [tdwg-content] Darwin Core: proposed news terms
for<br>
>> expressing sample data<br>
>> To: ?amonn ? Tuama [GBIF] <<a moz-do-not-send="true"
href="mailto:eotuama@gbif.org" target="_blank">eotuama@gbif.org</a>><br>
>> Cc: TDWG Content Mailing List <<a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a>><br>
>> Message-ID:<br>
>><br>
<CAFSW8xkx7uRP9PC2g3=<a moz-do-not-send="true"
href="mailto:JT_VJanqujH8nPXoz8GXwh%2BJwKw5Ccw@mail.gmail.com"
target="_blank">JT_VJanqujH8nPXoz8GXwh+JwKw5Ccw@mail.gmail.com</a>><br>
>> Content-Type: text/plain; charset="utf-8"<br>
>><br>
>> This proposal is treading on ground that is quite similar to
other<br>
>> observations and measurements standards for data exchange that
are<br>
already<br>
>> mature, in particular:<br>
>><br>
>> * OGC Observations and Measurements (<br>
>> <a moz-do-not-send="true"
href="http://www.opengeospatial.org/standards/om" target="_blank">http://www.opengeospatial.org/standards/om</a>)<br>
>> * Extensible Observation Ontology (OBOE;<br>
>> <a moz-do-not-send="true"
href="https://semtools.ecoinformatics.org/oboe" target="_blank">https://semtools.ecoinformatics.org/oboe</a>)<br>
>><br>
>> The former is a standard and broadly deployed, whereas the
latter is part<br>
>> of a research program in the use of ontologies for
measurements. Through<br>
>> collaboration between the two projects, they've been modified
to be<br>
>> reasonably isomorphic, but O&M uses an XML serialization
while OBOE uses<br>
an<br>
>> OWL-DL serialization. They largely express the same
measurements and<br>
>> sampling model once one gets beyond the terminology
differences.<br>
>><br>
>> So, I'm wondering if it make much sense to extend Darwin Core,
which is<br>
at<br>
>> heart an Occurrence exchange syntax, into this measurements
area that is<br>
>> well represented by these other existing specifications? I'm
curious to<br>
>> hear why people would even want to do this. And if we do go
down this<br>
>> path, won't we just end up with a new syntax that does
essentially what<br>
O&M<br>
>> and OBOE do now?<br>
>><br>
>> Matt<br>
>><br>
>><br>
>><br>
>> On Thu, Aug 21, 2014 at 12:22 AM, ?amonn ? Tuama [GBIF]<br>
<<a moz-do-not-send="true" href="mailto:eotuama@gbif.org"
target="_blank">eotuama@gbif.org</a>><br>
>> wrote:<br>
>><br>
>> > Hi Rob, Anne, Rich,<br>
>> ><br>
>> ><br>
>> ><br>
>> > I think Markus has answered your question as to why we
opted for an<br>
Event<br>
>> > core which is being used in the sense described by Anne
and Rich. For<br>
any<br>
>> > event, you can have a list of species in an Occurrence
extension and<br>
for<br>
>> > each species, you can include quantity and quantityType,
e.g., biomass,<br>
>> > etc. The proposed term eventSeriesID was intended for
linking together<br>
>> > related events, although it now looks like parentEventID
might be a<br>
better,<br>
>> > more flexible term. The measurementOrFact extension is a
good fit for<br>
>> > capturing environmental information relating to an event.
See, e.g.,<br>
the<br>
>> > Gialova Lagoon brackish water invertebrate test data set
[1] where a<br>
set<br>
>> > of 18 environmental variables, including temp, pH, Rdx,
particulate<br>
organic<br>
>> > matter, dissolved oxygen, salinity, chlorophyll-a were
measured for<br>
each<br>
>> > sampling station-sampling period combination. An example
mapping is:<br>
>> ><br>
>> ><br>
>> ><br>
>> > Id measurementType measurementValue<br>
>> > measurementUnit measurementRemarks<br>
>> ><br>
>> > IA Tmp (sed) 21.5<br>
>> > degree C
Tmp<br>
>> > (sed): temperature at the bottom surface<br>
>> ><br>
>> ><br>
>> ><br>
>> > **Controlled vocabularies**<br>
>> ><br>
>> > Ideally, the values for samplingUnit and quantityType
would be selected<br>
>> > from controlled vocabularies. This is, effectively, what
we do by<br>
>> > presenting a small list of values in a drop-down menu.
The current<br>
values<br>
>> > are what we derived for example data sets and discussion
but they can<br>
>> > undoubtedly be extended and improved.<br>
>> ><br>
>> ><br>
>> ><br>
>> > We capture ?bucket? type measures through a combination of<br>
samplingEffort,<br>
>> > samplingGeometry and samplingUnit. For example, a pitfall
trap (in a<br>
point<br>
>> > location) left out for 16 days might have samplingEffort:
16,<br>
>> > samplingGeometry: point and samplingUnit: day. Three m^2
quadrats in a<br>
>> > shore survey might have samplingEffort: 3,
samplingGeometry: area and<br>
>> > samplingUnit: m^2.<br>
>> ><br>
>> ><br>
>> ><br>
>> > It would be very useful to see your compilation of scope,
effort and<br>
>> > completeness measures to see if we can express them in
our model and/or<br>
if<br>
>> > we need to reconsider our approach.<br>
>> ><br>
>> ><br>
>> ><br>
>> > ?amonn<br>
>> ><br>
>> ><br>
>> ><br>
>> > [1] <a moz-do-not-send="true"
href="http://eubon-ipt.gbif.org/resource.do?r=ionian-brackish-lagoon"
target="_blank">http://eubon-ipt.gbif.org/resource.do?r=ionian-brackish-lagoon</a><br>
>> ><br>
>> ><br>
>> ><br>
>> > *From:* <a moz-do-not-send="true"
href="mailto:tdwg-content-bounces@lists.tdwg.org" target="_blank">tdwg-content-bounces@lists.tdwg.org</a>
[mailto:<br>
>> > <a moz-do-not-send="true"
href="mailto:tdwg-content-bounces@lists.tdwg.org" target="_blank">tdwg-content-bounces@lists.tdwg.org</a>]
*On Behalf Of *Markus D?ring<br>
>> > *Sent:* 20 August 2014 23:47<br>
>> > *To:* Robert Guralnick<br>
>> ><br>
>> > *Cc:* TDWG Content Mailing List<br>
>> > *Subject:* Re: [tdwg-content] Darwin Core: proposed news
terms for<br>
>> > expressing sample data<br>
>> ><br>
>> ><br>
>> ><br>
>> > Rob,<br>
>> ><br>
>> ><br>
>> ><br>
>> > this proposal if for monitoring surveys really, not to be
confused with<br>
>> > material samples like environmental or tissue samples
which have a<br>
distinct<br>
>> > new dwc class MaterialSample.<br>
>> ><br>
>> ><br>
>> ><br>
>> > We tend to overload the term sampling a lot and it helps
treating<br>
material<br>
>> > samples different from pure observational "sampling".
That is why the<br>
>> > existing Event class was used as the core and classic
Occurrence<br>
records as<br>
>> > extensions. A classic example is a vegetation survey
where each plot<br>
>> > represents an Event record and each recorded species in
that plot will<br>
be<br>
>> > an Occurrence extension record with a given quantity.
Darwin Core<br>
already<br>
>> > offers individualCount to specify quantity, but it is a
very specific<br>
way<br>
>> > of measuring "abundance" restricted to only some use
cases. Abiotic<br>
>> > measurements about the plot (e.g. soil type, pH,
temperature) can be<br>
>> > published using the measurements or facts extension
linked to the Event<br>
>> > core.<br>
>> ><br>
>> ><br>
>> ><br>
>> > Markus<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > On 20 Aug 2014, at 20:08, Robert Guralnick<br>
<<a moz-do-not-send="true"
href="mailto:Robert.Guralnick@colorado.edu" target="_blank">Robert.Guralnick@colorado.edu</a>><br>
>> > wrote:<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > Anne -- I don't know the answers! These are questions
for Eamonn. I<br>
>> > would presume that a sample could be a jumble of species
or even just<br>
water<br>
>> > or soil samples, and biomass would refer to that sample -
but maybe<br>
that<br>
>> > isn't a use case being considered? The examples given in
the longer<br>
>> > document all link an event_id to species name and some
measure of<br>
quantity<br>
>> > for that species (to the species, not an individual
specimen), so I<br>
assume<br>
>> > that is the prevailing (or only) case?<br>
>> ><br>
>> > Best, Rob<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > On Wed, Aug 20, 2014 at 11:56 AM, Anne Thessen <<a
moz-do-not-send="true" href="mailto:annethessen@gmail.com"
target="_blank">annethessen@gmail.com</a>><br>
>> > wrote:<br>
>> ><br>
>> > Hi Rob<br>
>> > I would like to respond to your item number 2.<br>
>> > From my perspective, I deal with lots of published
descriptions of<br>
taxa.<br>
>> > The text might say something like "I saw species A in the
Chesapeake<br>
Bay,<br>
>> > the Adriatic Sea and the Indian Ocean and the biomass is
5 - 9 grams".<br>
The<br>
>> > biomass range obviously corresponds to at least three
different<br>
>> > occurrences, but how to divide the biomass data? I would
love to be<br>
able to<br>
>> > have an *event* to attach it all to. There is almost two
different<br>
levels<br>
>> > of events - a sampling event and a "study event". The
"study event"<br>
would<br>
>> > correspond to the type of event I would like to use in
the above<br>
example.<br>
>> > It may not be ideal, but for the old literature that
might be the best<br>
we<br>
>> > can do.<br>
>> > I have to admit that I don't know enough about trawl data
to understand<br>
>> > why an event core would be a problem. It seems that the
trawl would be<br>
an<br>
>> > event and each biomass measure (of each fish) would be
attached to a<br>
>> > separate occurrence which is attached to that event. Am I
understanding<br>
>> > this wrong?<br>
>> > btw - I found a workaround for the example I gave, so
it's not<br>
impossible<br>
>> > to model with the current structure....<br>
>> > Anne<br>
>> ><br>
>> ><br>
>> ><br>
>> > On 8/20/2014 1:16 PM, Robert Guralnick wrote:<br>
>> ><br>
>> ><br>
>> ><br>
>> > ?amonn et al. --- Thanks for the clarifications. I think
these help a<br>
ton<br>
>> > but it raises a couple more questions for me.<br>
>> ><br>
>> ><br>
>> ><br>
>> > 1) I am surprised that you plan to use of
MeasurementorFact extension<br>
in<br>
>> > relation to the Event core, which seems like a novel (or
perhaps<br>
awkward or<br>
>> > unintended?) mechanism for capturing environmental data,
but the same<br>
>> > extension was not be seen as relevant for describing
samples? Can you<br>
>> > explain more about the thinking there?<br>
>> ><br>
>> ><br>
>> ><br>
>> > 2) There may be a subtle issue here extending "Event" to
be more what<br>
you<br>
>> > call a "Sampling Event Core". My read of this is that
Darwin Core<br>
serves<br>
>> > as a way to deal with point occurrences and Event
reflects the context<br>
of a<br>
>> > single capture event (whether a single observation, or a
bulk sample<br>
>> > capture). The changes recommended seem to dramatically
extend and<br>
change<br>
>> > that meaning? Its simply a question that I don't have
answer to, but<br>
is<br>
>> > Darwin Core, the right vehicle to start capturing
repeated measures of<br>
>> > biomass values from trawls? I don't have answer but
man, terms like<br>
>> > quantityType (as a property of occurrence?) give me pause.<br>
>> ><br>
>> ><br>
>> ><br>
>> > 3) Is Sampling Unit a controlled vocabulary? For another
project, I<br>
have<br>
>> > looked through - and captured scope, effort and
completeness measures<br>
from<br>
>> > - a large number of published biotic area inventories.
The vast<br>
majorities<br>
>> > of these are measured in units like bucket hours, or trap
nights. Is a<br>
>> > "bucket" part of SamplingGeometry or Sampling Unit? I'd
be happy to<br>
send<br>
>> > along all the many examples of how biotic inventories of
an area are<br>
>> > completed and perhaps it might be good to see how those
might be<br>
>> > represented using the terms you are proposing?<br>
>> ><br>
>> ><br>
>> ><br>
>> > Best, Rob<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > On Wed, Aug 20, 2014 at 10:16 AM, Richard Pyle<br>
<<a moz-do-not-send="true" href="mailto:deepreef@bishopmuseum.org"
target="_blank">deepreef@bishopmuseum.org</a>><br>
>> > wrote:<br>
>> ><br>
>> > Same here ? Events are central to the work that we do.<br>
>> ><br>
>> ><br>
>> ><br>
>> > Aloha,<br>
>> ><br>
>> > Rich<br>
>> ><br>
>> ><br>
>> ><br>
>> > *From:* <a moz-do-not-send="true"
href="mailto:tdwg-content-bounces@lists.tdwg.org" target="_blank">tdwg-content-bounces@lists.tdwg.org</a>
[mailto:<br>
>> > <a moz-do-not-send="true"
href="mailto:tdwg-content-bounces@lists.tdwg.org" target="_blank">tdwg-content-bounces@lists.tdwg.org</a>]
*On Behalf Of *Anne Thessen<br>
>> > *Sent:* Wednesday, August 20, 2014 2:59 AM<br>
>> > *To:* <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>> ><br>
>> ><br>
>> > *Subject:* Re: [tdwg-content] Darwin Core: proposed news
terms for<br>
>> > expressing sample data<br>
>> ><br>
>> ><br>
>> ><br>
>> > Hello<br>
>> > I would just like to comment on *event core*.<br>
>> > I've been doing a lot of work translating published data
into Darwin<br>
Core.<br>
>> > During that process I've wished several times that I
could use Event as<br>
>> > core. I am happy to hear about that proposed change. It
will make it<br>
easier<br>
>> > to model the data I am working with.<br>
>> > Anne<br>
>> ><br>
>> > On 8/20/2014 7:04 AM, ?amonn ? Tuama [GBIF] wrote:<br>
>> ><br>
>> > Hi Rob,<br>
>> ><br>
>> ><br>
>> ><br>
>> > Thank you for the feedback. I have tried to address the
two main issues<br>
>> > you raise below. At the outset, I would like to emphasise
that much of<br>
this<br>
>> > work is taking place in the context of the EU BON project
which<br>
includes a<br>
>> > task on developing/enhancing tools and standards for data
sharing with<br>
a<br>
>> > particular focus on the IPT for publishing sample-based
data. So, we<br>
were<br>
>> > constrained by the need to publish sample-based data sets
in the Darwin<br>
>> > Core Archive format and to demonstrate practical
application using a<br>
>> > working prototype. When the discussion on the TDWG list
faded out, we<br>
took<br>
>> > it to our EU BON partners whose requirements were
essential input to<br>
>> > further development. We recognise that these discussions
took place<br>
away<br>
>> > from TDWG (although the TDWG/EU BON contributors
overlapped) and this<br>
is<br>
>> > the reason we are presenting the outcomes here for
further<br>
consideration.<br>
>> ><br>
>> ><br>
>> ><br>
>> > **Event core**<br>
>> ><br>
>> > As the SIGS report indicated, sample data can be modelled
in Darwin<br>
Core<br>
>> > Archives using either Occurrence or Event as core. This
was the<br>
starting<br>
>> > point for our evaluation but as things progressed the
data wrangling<br>
pushed<br>
>> > the model back towards the Event core. We actually went
through the<br>
>> > exercise of mapping multiple test datasets in an
iterative process<br>
spanning<br>
>> > several months' work. In the end, we found that using an
Event core<br>
better<br>
>> > matched the typical sample data we were dealing with,
allowing use of a<br>
>> > measurement-or-fact extension to be included for the
efficient<br>
expression<br>
>> > of environmental information associated with the event.
The choice<br>
comes<br>
>> > down to an Occurrence core or an Event core + Occurrence
extension. In<br>
both<br>
>> > cases, the true observation records are Occurrences. The
big difference<br>
is<br>
>> > what type the core has and therefore to which kind of
records you can<br>
>> > attach further facts and extra information with DwC-A
extensions. Many<br>
>> > sampling datasets have very rich information about the
site and event,<br>
so<br>
>> > it is very natural to hang facts from an Event core. When
picking the<br>
>> > Occurrence core those facts would have to be repeated for
each and<br>
every<br>
>> > occurrence record. Moreover, our approach doesn?t stop
anyone from<br>
using<br>
>> > the Occurrence core if they so wish. This just provides a
different<br>
option<br>
>> > for datasets that better fit an Event core model.<br>
>> ><br>
>> ><br>
>> ><br>
>> > I want to stress that we are not building a ?specific IPT
version? to<br>
>> > support an Event core but, rather, we adapted the IPT so
that it can be<br>
>> > configured to support any generic ?core + extension?
format to enable<br>
its<br>
>> > use for exploration of more data formats. This is part
of the core<br>
>> > codebase and there were no custom forks of the IPT for
this work. Our<br>
view<br>
>> > at GBIF is that if there are significant numbers of data
publishers who<br>
are<br>
>> > keen to adopt, promote and use a (any) format, and the
tools can be<br>
>> > configured to do so, then we should support it, and, if
necessary, use<br>
a<br>
>> > custom namespace.<br>
>> ><br>
>> ><br>
>> ><br>
>> > **New terms around abundance**<br>
>> ><br>
>> > Yes, the discussion on TDWG did fade out but it was clear
that the term<br>
>> > ?abundance? as recommended by the SIGS report (along with<br>
>> > abundanceAsPercent) was confusing many when we were
looking for term(s)<br>
>> > that reported quantitative measures of organisms in a
sample. It also<br>
>> > became clear we would need to be able to state the type
of quantity<br>
being<br>
>> > measured. An alternative suggestion for using the
MeasurementsOrFact<br>
class<br>
>> > was immediately shot down.<br>
>> ><br>
>> > As some of our main use cases were coming from the EU BON
project,<br>
>> > discussion shifted to that forum and consensus formed
about the<br>
currently<br>
>> > proposed terms. It was within this group that the
additional terms<br>
>> > (samplingGeometry, samplingUnit, eventSeriesID) were
proposed and where<br>
we<br>
>> > began testing with sample data sets.<br>
>> ><br>
>> ><br>
>> ><br>
>> > Best regards,<br>
>> ><br>
>> > ?amonn<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > *From:* <a moz-do-not-send="true"
href="mailto:robgur@gmail.com" target="_blank">robgur@gmail.com</a>
[mailto:<a moz-do-not-send="true" href="mailto:robgur@gmail.com"
target="_blank">robgur@gmail.com</a> <<a moz-do-not-send="true"
href="mailto:robgur@gmail.com" target="_blank">robgur@gmail.com</a>>]<br>
*On<br>
>> > Behalf Of *Robert Guralnick<br>
>> > *Sent:* 19 August 2014 16:56<br>
>> > *To:* ?amonn ? Tuama [GBIF]<br>
>> > *Cc:* TDWG Content Mailing List<br>
>> > *Subject:* Re: [tdwg-content] Darwin Core: proposed news
terms for<br>
>> > expressing sample data<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > Hi ?amonn --- I am curious about the outcomes presented
in the SIGS<br>
>> > paper, in particular, this portion of the paper:<br>
>> ><br>
>> ><br>
>> ><br>
>> > "Solutions without introducing an event core in Darwin
Core Archives:<br>
>> > During the review of the solutions for the uses cases,
it became<br>
apparent<br>
>> > that either model could be applied to every use case. The
core and<br>
>> > extensions bore a complementary relationship and between
them could<br>
express<br>
>> > all the required information. The core simply provided
the central<br>
anchor<br>
>> > in the star schema from which to join the additional
information.<br>
>> > Therefore, using the Occurrence core, well established in
the GBIF<br>
network<br>
>> > through uptake of the IPT, seemed more appropriate than
inventing<br>
>> > CollectingEvent as an additional core type."<br>
>> ><br>
>> ><br>
>> ><br>
>> > That SIGS paper has John Wieczorek and you both as
authors,<br>
including<br>
>> > many luminaries across the biodiversity standards
spectrum. Given the<br>
>> > above, its curious to see the EventCore come back again,
along with a<br>
>> > specific IPT version to support it.<br>
>> ><br>
>> ><br>
>> ><br>
>> > So I see two issues, conflated, in this post you just
made. One is<br>
>> > the need for an EventCore at all, and the nature of
relating Event and<br>
>> > Occurrence/Material Sample. The second is the
introduction of new<br>
terms,<br>
>> > which seemingly have arrived after debate on similar
terms - but framed<br>
>> > around abundance - stalled a year ago. To my mind, these
both require<br>
some<br>
>> > further discussion, because I don't (necessarily) see
TDWG community<br>
>> > coherence around either issue?<br>
>> ><br>
>> ><br>
>> ><br>
>> > Best, Rob<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > On Tue, Aug 19, 2014 at 6:11 AM, ?amonn ? Tuama [GBIF]<br>
<<a moz-do-not-send="true" href="mailto:eotuama@gbif.org"
target="_blank">eotuama@gbif.org</a>><br>
>> > wrote:<br>
>> ><br>
>> > Dear All,<br>
>> ><br>
>> ><br>
>> ><br>
>> > GBIF is committed to exploring ways in which the IPT and
Darwin Core<br>
>> > Archive format can be extended for publishing
sample-based data sets.<br>
In<br>
>> > association with the EU BON project [1], a customised
version of the<br>
IPT<br>
>> > [2] has been deployed to test this using a special type
of Darwin Core<br>
>> > Archive in which the core is an ?Event? with associated
taxon<br>
occurrences<br>
>> > in an ?Occurrence? extension.<br>
>> ><br>
>> ><br>
>> ><br>
>> > The Darwin Core vocabulary already provides a rich set of
terms with<br>
many<br>
>> > relevant for describing sample-based data. Synthesising
several sources<br>
of<br>
>> > input (GBIF organised workshop on sample data, May 2013
[3],<br>
discussions on<br>
>> > the TDWG mailing list in late 2013; internal discussion
among EU BON<br>
>> > project partners), five new terms relating to sample data
were<br>
identified<br>
>> > as essential. The complete model including these new
terms are fully<br>
>> > described with examples in the online document
?Publishing sample data<br>
>> > using the GBIF IPT? [4].<br>
>> ><br>
>> ><br>
>> ><br>
>> > As a first step towards ratification, we would like to
register the new<br>
>> > terms in the DwC Google Code tracker [5] if there are no
major<br>
objections<br>
>> > on this list. The five terms are:<br>
>> ><br>
>> ><br>
>> ><br>
>> > 1. *quantity*: the number or enumeration value of
the quantityType<br>
>> > (e.g., individuals, biomass, biovolume,
BraunBlanquetScale) per<br>
>> > samplingUnit or a percentage measure recorded for the
sample.<br>
>> ><br>
>> ><br>
>> ><br>
>> > 2. *quantityType*: : the entity being referred to
by quantity,<br>
>> > e.g., individuals, biomass, %species, scale type.<br>
>> ><br>
>> ><br>
>> ><br>
>> > 3. *samplingGeometry*: an indication of what kind of
space was<br>
>> > sampled; select from point, line, area or volume.<br>
>> ><br>
>> ><br>
>> ><br>
>> > 4. *samplingUnit*: the unit of measurement used for
reporting the<br>
>> > quantity in the sample, e.g., minute, hour, day, metre,
metre^2,<br>
metre^3.<br>
>> > It is combined with quantity and quantityType to provide
the complete<br>
>> > measurement, e.g., 9 individuals per day, 4 biomass-gm
per metre^2.<br>
>> ><br>
>> ><br>
>> ><br>
>> > 5. *eventSeriesID*: an identifier for a set of
events that are<br>
>> > associated in some way, e.g., a monitoring series; may be
a global<br>
unique<br>
>> > identifier or an identifier specific to the series.<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > Best regards,<br>
>> ><br>
>> ><br>
>> ><br>
>> > ?amonn<br>
>> ><br>
>> ><br>
>> ><br>
>> > [1] <a moz-do-not-send="true" href="http://eubon.eu"
target="_blank">http://eubon.eu</a><br>
>> ><br>
>> > [2] <a moz-do-not-send="true"
href="http://eubon-ipt.gbif.org" target="_blank">http://eubon-ipt.gbif.org</a><br>
>> ><br>
>> > [3]<br>
>> ><br>
<a moz-do-not-send="true"
href="http://www.standardsingenomics.org/index.php/sigen/article/view/sigs.4898640"
target="_blank">http://www.standardsingenomics.org/index.php/sigen/article/view/sigs.4898640</a><br>
>> ><br>
>> > [4] <a moz-do-not-send="true"
href="http://links.gbif.org/sample_data_model" target="_blank">http://links.gbif.org/sample_data_model</a><br>
>> ><br>
>> > [5] <a moz-do-not-send="true"
href="https://code.google.com/p/darwincore/issues/list" target="_blank">https://code.google.com/p/darwincore/issues/list</a><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > ____________________________________________________<br>
>> ><br>
>> > *?amonn ? Tuama, M.Sc., Ph.D. (<a moz-do-not-send="true"
href="mailto:eotuama@gbif.org" target="_blank">eotuama@gbif.org</a>
<<a moz-do-not-send="true" href="mailto:eotuama@gbif.org"
target="_blank">eotuama@gbif.org</a>>), *<br>
>> ><br>
>> > *Senior Programme Officer for Interoperability, *<br>
>> ><br>
>> > *Global Biodiversity Information Facility Secretariat, *<br>
>> ><br>
>> > *Universitetsparken 15, DK-2100, Copenhagen ?, DENMARK*<br>
>> ><br>
>> > *Phone: <a moz-do-not-send="true"
href="tel:%2B45%203532%201494" target="_blank">+45 3532 1494</a>
<%2B45%203532%201494>; Fax: <a moz-do-not-send="true"
href="tel:%2B45%203532%201480" target="_blank">+45 3532 1480</a><br>
>> > <%2B45%203532%201480>*<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> > tdwg-content mailing list<br>
>> > <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>> > <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> ><br>
>> > tdwg-content mailing list<br>
>> ><br>
>> > <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>> ><br>
>> > <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>> ><br>
>> ><br>
>> ><br>
>> > --<br>
>> ><br>
>> > Anne E. Thessen, Ph.D.<br>
>> ><br>
>> > The Data Detektiv, Owner and Founder<br>
>> ><br>
>> > Ronin Institute, Research Scholar<br>
>> ><br>
>> > <a moz-do-not-send="true" href="tel:443.225.9185"
target="_blank">443.225.9185</a><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> > tdwg-content mailing list<br>
>> > <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>> > <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > --<br>
>> ><br>
>> > Anne E. Thessen, Ph.D.<br>
>> ><br>
>> > The Data Detektiv, Owner and Founder<br>
>> ><br>
>> > Ronin Institute, Research Scholar<br>
>> ><br>
>> > <a moz-do-not-send="true" href="tel:443.225.9185"
target="_blank">443.225.9185</a><br>
>> ><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> > tdwg-content mailing list<br>
>> > <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>> > <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>> ><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> > tdwg-content mailing list<br>
>> > <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>> > <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>> ><br>
>> ><br>
>> -------------- next part --------------<br>
>> An HTML attachment was scrubbed...<br>
>> URL:<br>
<a moz-do-not-send="true"
href="http://lists.tdwg.org/pipermail/tdwg-content/attachments/20140821/4b338606/attachment-0001.html"
target="_blank">http://lists.tdwg.org/pipermail/tdwg-content/attachments/20140821/4b338606/a<br>
ttachment-0001.html</a><br>
>><br>
>> ------------------------------<br>
>><br>
>> Message: 2<br>
>> Date: Fri, 22 Aug 2014 08:54:07 +0200<br>
>> From: "Donald Hobern [GBIF]" <<a moz-do-not-send="true"
href="mailto:dhobern@gbif.org" target="_blank">dhobern@gbif.org</a>><br>
>> Subject: Re: [tdwg-content] Darwin Core: proposed news terms
for<br>
>> expressing sample data<br>
>> To: "'Matt Jones'" <<a moz-do-not-send="true"
href="mailto:jones@nceas.ucsb.edu" target="_blank">jones@nceas.ucsb.edu</a>>,
'?amonn ? Tuama [GBIF]'<br>
>> <<a moz-do-not-send="true"
href="mailto:eotuama@gbif.org" target="_blank">eotuama@gbif.org</a>><br>
>> Cc: 'TDWG Content Mailing List' <<a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a>><br>
>> Message-ID: <003401cfbdd5$de52b640$9af822c0$@<a
moz-do-not-send="true" href="http://gbif.org" target="_blank">gbif.org</a>><br>
>> Content-Type: text/plain; charset="utf-8"<br>
>><br>
>> Hi Matt,<br>
>><br>
>><br>
>><br>
>> I?ll take the chance to make a few quick comments here,
because I believe<br>
that this work is of massive importance.<br>
>><br>
>><br>
>><br>
>> Clearly DwC should avoid trying to duplicate well-standardised
models and<br>
protocols. However at the same time, there is enormous value for
producers<br>
and consumers of DwC to benefit from richer data on the events and
methods<br>
associated with individual species occurrences. I have never seen DwC
as<br>
purely an Occurrence exchange syntax. I see it (from GBIF?s standpoint)<br>
more closely as a mechanism for diverse parties to pool the evidence
they<br>
have for the occurrence of any species including associated information<br>
and/or actionable links to associated information. Users coming from
this<br>
perspective certainly need (and are demanding) access to all the
evidence<br>
that can be mobilized to serve as supporting evidence and they also
need the<br>
ability to understand the significance of these records. Abundance<br>
measures, levels of effort, use of consistent methods and redetection of<br>
individual organisms are all part of this. DwC should be able to
transmit<br>
as much data as publishers cho<br>
>> ose to share on such aspects as part of their publishing of
DwC. Users<br>
of DwC carrying out species modeling, threat assessment or community<br>
analyses will benefit from rapid ways to filter data for those which
derive<br>
from standardized sampling events, to understand relative abundance
within<br>
samples, etc. Many publishers of DwC are currently sharing
stripped-down<br>
subsets of data and wish to give more information on these points.
Users<br>
are certainly demanding it.<br>
>><br>
>><br>
>><br>
>> The challenge is finding the sweet spot, the achievable,
non-destructive<br>
overlap between DwC and the proper domain of models better designed to<br>
handle the representation of complex systems outside DwC?s current
domain.<br>
If this is done correctly, there should be paths that enable us to
generate<br>
O&E (and maybe OBOE) compatible data from data that publishers only
serve as<br>
augmented DwC.<br>
>><br>
>><br>
>><br>
>> I?ll also note that this has been a prominent area of
discussion now for<br>
several years. Many of us believe strongly that this is one of the most<br>
important ways in which we need to close arbitrary gaps between data
silos.<br>
It?s a prominent part of the GBIF work programme for 2014-2016.<br>
>><br>
>><br>
>><br>
>> Very best wishes,<br>
>><br>
>><br>
>><br>
>> Donald<br>
>><br>
>><br>
>><br>
>> From: <a moz-do-not-send="true"
href="mailto:tdwg-content-bounces@lists.tdwg.org" target="_blank">tdwg-content-bounces@lists.tdwg.org</a><br>
[mailto:<a moz-do-not-send="true"
href="mailto:tdwg-content-bounces@lists.tdwg.org" target="_blank">tdwg-content-bounces@lists.tdwg.org</a>]
On Behalf Of Matt Jones<br>
>> Sent: Friday, August 22, 2014 4:52 AM<br>
>> To: ?amonn ? Tuama [GBIF]<br>
>> Cc: TDWG Content Mailing List<br>
>> Subject: Re: [tdwg-content] Darwin Core: proposed news terms
for<br>
expressing sample data<br>
>><br>
>><br>
>><br>
>> This proposal is treading on ground that is quite similar to
other<br>
observations and measurements standards for data exchange that are
already<br>
mature, in particular:<br>
>><br>
>><br>
>><br>
>> * OGC Observations and Measurements<br>
(<a moz-do-not-send="true"
href="http://www.opengeospatial.org/standards/om" target="_blank">http://www.opengeospatial.org/standards/om</a>)<br>
>><br>
>> * Extensible Observation Ontology (OBOE;<br>
<a moz-do-not-send="true"
href="https://semtools.ecoinformatics.org/oboe" target="_blank">https://semtools.ecoinformatics.org/oboe</a>)<br>
>><br>
>><br>
>><br>
>> The former is a standard and broadly deployed, whereas the
latter is part<br>
of a research program in the use of ontologies for measurements.
Through<br>
collaboration between the two projects, they've been modified to be<br>
reasonably isomorphic, but O&M uses an XML serialization while OBOE
uses an<br>
OWL-DL serialization. They largely express the same measurements and<br>
sampling model once one gets beyond the terminology differences.<br>
>><br>
>><br>
>><br>
>> So, I'm wondering if it make much sense to extend Darwin Core,
which is<br>
at heart an Occurrence exchange syntax, into this measurements area
that is<br>
well represented by these other existing specifications? I'm curious to<br>
hear why people would even want to do this. And if we do go down this
path,<br>
won't we just end up with a new syntax that does essentially what
O&M and<br>
OBOE do now?<br>
>><br>
>><br>
>><br>
>> Matt<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> On Thu, Aug 21, 2014 at 12:22 AM, ?amonn ? Tuama [GBIF]<br>
<<a moz-do-not-send="true" href="mailto:eotuama@gbif.org"
target="_blank">eotuama@gbif.org</a>> wrote:<br>
>><br>
>> Hi Rob, Anne, Rich,<br>
>><br>
>><br>
>><br>
>> I think Markus has answered your question as to why we opted
for an Event<br>
core which is being used in the sense described by Anne and Rich. For
any<br>
event, you can have a list of species in an Occurrence extension and for<br>
each species, you can include quantity and quantityType, e.g., biomass,
etc.<br>
The proposed term eventSeriesID was intended for linking together
related<br>
events, although it now looks like parentEventID might be a better, more<br>
flexible term. The measurementOrFact extension is a good fit for
capturing<br>
environmental information relating to an event. See, e.g., the Gialova<br>
Lagoon brackish water invertebrate test data set [1] where a set of 18<br>
environmental variables, including temp, pH, Rdx, particulate organic<br>
matter, dissolved oxygen, salinity, chlorophyll-a were measured for each<br>
sampling station-sampling period combination. An example mapping is:<br>
>><br>
>><br>
>><br>
>> Id measurementType measurementValue<br>
measurementUnit measurementRemarks<br>
>><br>
>> IA Tmp (sed) 21.5<br>
degree C Tmp (sed): temperature at the
bottom<br>
surface<br>
>><br>
>><br>
>><br>
>> *Controlled vocabularies*<br>
>><br>
>> Ideally, the values for samplingUnit and quantityType would be
selected<br>
from controlled vocabularies. This is, effectively, what we do by
presenting<br>
a small list of values in a drop-down menu. The current values are what
we<br>
derived for example data sets and discussion but they can undoubtedly be<br>
extended and improved.<br>
>><br>
>><br>
>><br>
>> We capture ?bucket? type measures through a combination of<br>
samplingEffort, samplingGeometry and samplingUnit. For example, a
pitfall<br>
trap (in a point location) left out for 16 days might have
samplingEffort:<br>
16, samplingGeometry: point and samplingUnit: day. Three m^2 quadrats
in a<br>
shore survey might have samplingEffort: 3, samplingGeometry: area and<br>
samplingUnit: m^2.<br>
>><br>
>><br>
>><br>
>> It would be very useful to see your compilation of scope,
effort and<br>
completeness measures to see if we can express them in our model and/or
if<br>
we need to reconsider our approach.<br>
>><br>
>><br>
>><br>
>> ?amonn<br>
>><br>
>><br>
>><br>
>> [1] <a moz-do-not-send="true"
href="http://eubon-ipt.gbif.org/resource.do?r=ionian-brackish-lagoon"
target="_blank">http://eubon-ipt.gbif.org/resource.do?r=ionian-brackish-lagoon</a><br>
>><br>
>><br>
>><br>
>> From: <a moz-do-not-send="true"
href="mailto:tdwg-content-bounces@lists.tdwg.org" target="_blank">tdwg-content-bounces@lists.tdwg.org</a><br>
[mailto:<a moz-do-not-send="true"
href="mailto:tdwg-content-bounces@lists.tdwg.org" target="_blank">tdwg-content-bounces@lists.tdwg.org</a>]
On Behalf Of Markus D?ring<br>
>> Sent: 20 August 2014 23:47<br>
>> To: Robert Guralnick<br>
>><br>
>><br>
>> Cc: TDWG Content Mailing List<br>
>> Subject: Re: [tdwg-content] Darwin Core: proposed news terms
for<br>
expressing sample data<br>
>><br>
>><br>
>><br>
>> Rob,<br>
>><br>
>><br>
>><br>
>> this proposal if for monitoring surveys really, not to be
confused with<br>
material samples like environmental or tissue samples which have a
distinct<br>
new dwc class MaterialSample.<br>
>><br>
>><br>
>><br>
>> We tend to overload the term sampling a lot and it helps
treating<br>
material samples different from pure observational "sampling". That is
why<br>
the existing Event class was used as the core and classic Occurrence
records<br>
as extensions. A classic example is a vegetation survey where each plot<br>
represents an Event record and each recorded species in that plot will
be an<br>
Occurrence extension record with a given quantity. Darwin Core already<br>
offers individualCount to specify quantity, but it is a very specific
way of<br>
measuring "abundance" restricted to only some use cases. Abiotic<br>
measurements about the plot (e.g. soil type, pH, temperature) can be<br>
published using the measurements or facts extension linked to the Event<br>
core.<br>
>><br>
>><br>
>><br>
>> Markus<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> On 20 Aug 2014, at 20:08, Robert Guralnick<br>
<<a moz-do-not-send="true"
href="mailto:Robert.Guralnick@colorado.edu" target="_blank">Robert.Guralnick@colorado.edu</a>>
wrote:<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> Anne -- I don't know the answers! These are questions for
Eamonn. I<br>
would presume that a sample could be a jumble of species or even just
water<br>
or soil samples, and biomass would refer to that sample - but maybe that<br>
isn't a use case being considered? The examples given in the longer<br>
document all link an event_id to species name and some measure of
quantity<br>
for that species (to the species, not an individual specimen), so I
assume<br>
that is the prevailing (or only) case?<br>
>><br>
>> Best, Rob<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> On Wed, Aug 20, 2014 at 11:56 AM, Anne Thessen <<a
moz-do-not-send="true" href="mailto:annethessen@gmail.com"
target="_blank">annethessen@gmail.com</a>><br>
wrote:<br>
>><br>
>> Hi Rob<br>
>> I would like to respond to your item number 2.<br>
>> >From my perspective, I deal with lots of published
descriptions of taxa.<br>
The text might say something like "I saw species A in the Chesapeake
Bay,<br>
the Adriatic Sea and the Indian Ocean and the biomass is 5 - 9 grams".
The<br>
biomass range obviously corresponds to at least three different
occurrences,<br>
but how to divide the biomass data? I would love to be able to have an<br>
*event* to attach it all to. There is almost two different levels of
events<br>
- a sampling event and a "study event". The "study event" would
correspond<br>
to the type of event I would like to use in the above example. It may
not be<br>
ideal, but for the old literature that might be the best we can do.<br>
>> I have to admit that I don't know enough about trawl data to
understand<br>
why an event core would be a problem. It seems that the trawl would be
an<br>
event and each biomass measure (of each fish) would be attached to a<br>
separate occurrence which is attached to that event. Am I understanding
this<br>
wrong?<br>
>> btw - I found a workaround for the example I gave, so it's not
impossible<br>
to model with the current structure....<br>
>> Anne<br>
>><br>
>><br>
>><br>
>> On 8/20/2014 1:16 PM, Robert Guralnick wrote:<br>
>><br>
>><br>
>><br>
>> ?amonn et al. --- Thanks for the clarifications. I think
these help a<br>
ton but it raises a couple more questions for me.<br>
>><br>
>><br>
>><br>
>> 1) I am surprised that you plan to use of MeasurementorFact
extension<br>
in relation to the Event core, which seems like a novel (or perhaps
awkward<br>
or unintended?) mechanism for capturing environmental data, but the same<br>
extension was not be seen as relevant for describing samples? Can you<br>
explain more about the thinking there?<br>
>><br>
>><br>
>><br>
>> 2) There may be a subtle issue here extending "Event" to be
more what<br>
you call a "Sampling Event Core". My read of this is that Darwin Core<br>
serves as a way to deal with point occurrences and Event reflects the<br>
context of a single capture event (whether a single observation, or a
bulk<br>
sample capture). The changes recommended seem to dramatically extend
and<br>
change that meaning? Its simply a question that I don't have answer
to, but<br>
is Darwin Core, the right vehicle to start capturing repeated measures
of<br>
biomass values from trawls? I don't have answer but man, terms like<br>
quantityType (as a property of occurrence?) give me pause.<br>
>><br>
>><br>
>><br>
>> 3) Is Sampling Unit a controlled vocabulary? For another
project, I have<br>
looked through - and captured scope, effort and completeness measures
from -<br>
a large number of published biotic area inventories. The vast
majorities of<br>
these are measured in units like bucket hours, or trap nights. Is a<br>
"bucket" part of SamplingGeometry or Sampling Unit? I'd be happy to
send<br>
along all the many examples of how biotic inventories of an area are<br>
completed and perhaps it might be good to see how those might be
represented<br>
using the terms you are proposing?<br>
>><br>
>><br>
>><br>
>> Best, Rob<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> On Wed, Aug 20, 2014 at 10:16 AM, Richard Pyle<br>
<<a moz-do-not-send="true" href="mailto:deepreef@bishopmuseum.org"
target="_blank">deepreef@bishopmuseum.org</a>> wrote:<br>
>><br>
>> Same here ? Events are central to the work that we do.<br>
>><br>
>><br>
>><br>
>> Aloha,<br>
>><br>
>> Rich<br>
>><br>
>><br>
>><br>
>> From: <a moz-do-not-send="true"
href="mailto:tdwg-content-bounces@lists.tdwg.org" target="_blank">tdwg-content-bounces@lists.tdwg.org</a><br>
[mailto:<a moz-do-not-send="true"
href="mailto:tdwg-content-bounces@lists.tdwg.org" target="_blank">tdwg-content-bounces@lists.tdwg.org</a>]
On Behalf Of Anne Thessen<br>
>> Sent: Wednesday, August 20, 2014 2:59 AM<br>
>> To: <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>><br>
>><br>
>> Subject: Re: [tdwg-content] Darwin Core: proposed news terms
for<br>
expressing sample data<br>
>><br>
>><br>
>><br>
>> Hello<br>
>> I would just like to comment on *event core*.<br>
>> I've been doing a lot of work translating published data into
Darwin<br>
Core. During that process I've wished several times that I could use
Event<br>
as core. I am happy to hear about that proposed change. It will make it<br>
easier to model the data I am working with.<br>
>> Anne<br>
>><br>
>> On 8/20/2014 7:04 AM, ?amonn ? Tuama [GBIF] wrote:<br>
>><br>
>> Hi Rob,<br>
>><br>
>><br>
>><br>
>> Thank you for the feedback. I have tried to address the two
main issues<br>
you raise below. At the outset, I would like to emphasise that much of
this<br>
work is taking place in the context of the EU BON project which
includes a<br>
task on developing/enhancing tools and standards for data sharing with a<br>
particular focus on the IPT for publishing sample-based data. So, we
were<br>
constrained by the need to publish sample-based data sets in the Darwin
Core<br>
Archive format and to demonstrate practical application using a working<br>
prototype. When the discussion on the TDWG list faded out, we took it
to our<br>
EU BON partners whose requirements were essential input to further<br>
development. We recognise that these discussions took place away from
TDWG<br>
(although the TDWG/EU BON contributors overlapped) and this is the
reason we<br>
are presenting the outcomes here for further consideration.<br>
>><br>
>><br>
>><br>
>> *Event core*<br>
>><br>
>> As the SIGS report indicated, sample data can be modelled in
Darwin Core<br>
Archives using either Occurrence or Event as core. This was the starting<br>
point for our evaluation but as things progressed the data wrangling
pushed<br>
the model back towards the Event core. We actually went through the
exercise<br>
of mapping multiple test datasets in an iterative process spanning
several<br>
months' work. In the end, we found that using an Event core better
matched<br>
the typical sample data we were dealing with, allowing use of a<br>
measurement-or-fact extension to be included for the efficient
expression of<br>
environmental information associated with the event. The choice comes
down<br>
to an Occurrence core or an Event core + Occurrence extension. In both<br>
cases, the true observation records are Occurrences. The big difference
is<br>
what type the core has and therefore to which kind of records you can
attach<br>
further facts and extra information with DwC-A extensions. Many sampling<br>
datasets have very rich infor<br>
>> mation about the site and event, so it is very natural to
hang facts<br>
from an Event core. When picking the Occurrence core those facts would
have<br>
to be repeated for each and every occurrence record. Moreover, our
approach<br>
doesn?t stop anyone from using the Occurrence core if they so wish. This<br>
just provides a different option for datasets that better fit an Event
core<br>
model.<br>
>><br>
>><br>
>><br>
>> I want to stress that we are not building a ?specific IPT
version? to<br>
support an Event core but, rather, we adapted the IPT so that it can be<br>
configured to support any generic ?core + extension? format to enable
its<br>
use for exploration of more data formats. This is part of the core
codebase<br>
and there were no custom forks of the IPT for this work. Our view at
GBIF<br>
is that if there are significant numbers of data publishers who are
keen to<br>
adopt, promote and use a (any) format, and the tools can be configured
to do<br>
so, then we should support it, and, if necessary, use a custom
namespace.<br>
>><br>
>><br>
>><br>
>> *New terms around abundance*<br>
>><br>
>> Yes, the discussion on TDWG did fade out but it was clear that
the term<br>
?abundance? as recommended by the SIGS report (along with<br>
abundanceAsPercent) was confusing many when we were looking for term(s)
that<br>
reported quantitative measures of organisms in a sample. It also became<br>
clear we would need to be able to state the type of quantity being
measured.<br>
An alternative suggestion for using the MeasurementsOrFact class was<br>
immediately shot down.<br>
>><br>
>> As some of our main use cases were coming from the EU BON
project,<br>
discussion shifted to that forum and consensus formed about the
currently<br>
proposed terms. It was within this group that the additional terms<br>
(samplingGeometry, samplingUnit, eventSeriesID) were proposed and where
we<br>
began testing with sample data sets.<br>
>><br>
>><br>
>><br>
>> Best regards,<br>
>><br>
>> ?amonn<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> From: <a moz-do-not-send="true" href="mailto:robgur@gmail.com"
target="_blank">robgur@gmail.com</a> [mailto:<a moz-do-not-send="true"
href="mailto:robgur@gmail.com" target="_blank">robgur@gmail.com</a>]
On Behalf Of Robert<br>
Guralnick<br>
>> Sent: 19 August 2014 16:56<br>
>> To: ?amonn ? Tuama [GBIF]<br>
>> Cc: TDWG Content Mailing List<br>
>> Subject: Re: [tdwg-content] Darwin Core: proposed news terms
for<br>
expressing sample data<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> Hi ?amonn --- I am curious about the outcomes presented in
the SIGS<br>
paper, in particular, this portion of the paper:<br>
>><br>
>><br>
>><br>
>> "Solutions without introducing an event core in Darwin Core
Archives:<br>
During the review of the solutions for the uses cases, it became
apparent<br>
that either model could be applied to every use case. The core and<br>
extensions bore a complementary relationship and between them could
express<br>
all the required information. The core simply provided the central
anchor in<br>
the star schema from which to join the additional information.
Therefore,<br>
using the Occurrence core, well established in the GBIF network through<br>
uptake of the IPT, seemed more appropriate than inventing
CollectingEvent as<br>
an additional core type."<br>
>><br>
>><br>
>><br>
>> That SIGS paper has John Wieczorek and you both as authors,
including<br>
many luminaries across the biodiversity standards spectrum. Given the<br>
above, its curious to see the EventCore come back again, along with a<br>
specific IPT version to support it.<br>
>><br>
>><br>
>><br>
>> So I see two issues, conflated, in this post you just
made. One is<br>
the need for an EventCore at all, and the nature of relating Event and<br>
Occurrence/Material Sample. The second is the introduction of new
terms,<br>
which seemingly have arrived after debate on similar terms - but framed<br>
around abundance - stalled a year ago. To my mind, these both require
some<br>
further discussion, because I don't (necessarily) see TDWG community<br>
coherence around either issue?<br>
>><br>
>><br>
>><br>
>> Best, Rob<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> On Tue, Aug 19, 2014 at 6:11 AM, ?amonn ? Tuama [GBIF] <<a
moz-do-not-send="true" href="mailto:eotuama@gbif.org" target="_blank">eotuama@gbif.org</a>><br>
wrote:<br>
>><br>
>> Dear All,<br>
>><br>
>><br>
>><br>
>> GBIF is committed to exploring ways in which the IPT and
Darwin Core<br>
Archive format can be extended for publishing sample-based data sets. In<br>
association with the EU BON project [1], a customised version of the
IPT [2]<br>
has been deployed to test this using a special type of Darwin Core
Archive<br>
in which the core is an ?Event? with associated taxon occurrences in an<br>
?Occurrence? extension.<br>
>><br>
>><br>
>><br>
>> The Darwin Core vocabulary already provides a rich set of
terms with many<br>
relevant for describing sample-based data. Synthesising several sources
of<br>
input (GBIF organised workshop on sample data, May 2013 [3],
discussions on<br>
the TDWG mailing list in late 2013; internal discussion among EU BON
project<br>
partners), five new terms relating to sample data were identified as<br>
essential. The complete model including these new terms are fully
described<br>
with examples in the online document ?Publishing sample data using the
GBIF<br>
IPT? [4].<br>
>><br>
>><br>
>><br>
>> As a first step towards ratification, we would like to
register the new<br>
terms in the DwC Google Code tracker [5] if there are no major
objections on<br>
this list. The five terms are:<br>
>><br>
>><br>
>><br>
>> 1. quantity: the number or enumeration value of the
quantityType<br>
(e.g., individuals, biomass, biovolume, BraunBlanquetScale) per
samplingUnit<br>
or a percentage measure recorded for the sample.<br>
>><br>
>><br>
>><br>
>> 2. quantityType: : the entity being referred to by
quantity, e.g.,<br>
individuals, biomass, %species, scale type.<br>
>><br>
>><br>
>><br>
>> 3. samplingGeometry: an indication of what kind of space
was<br>
sampled; select from point, line, area or volume.<br>
>><br>
>><br>
>><br>
>> 4. samplingUnit: the unit of measurement used for
reporting the<br>
quantity in the sample, e.g., minute, hour, day, metre, metre^2,
metre^3.<br>
It is combined with quantity and quantityType to provide the complete<br>
measurement, e.g., 9 individuals per day, 4 biomass-gm per metre^2.<br>
>><br>
>><br>
>><br>
>> 5. eventSeriesID: an identifier for a set of events that
are<br>
associated in some way, e.g., a monitoring series; may be a global
unique<br>
identifier or an identifier specific to the series.<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> Best regards,<br>
>><br>
>><br>
>><br>
>> ?amonn<br>
>><br>
>><br>
>><br>
>> [1] <a moz-do-not-send="true" href="http://eubon.eu"
target="_blank">http://eubon.eu</a> <<a moz-do-not-send="true"
href="http://eubon.eu/" target="_blank">http://eubon.eu/</a>><br>
>><br>
>> [2] <a moz-do-not-send="true" href="http://eubon-ipt.gbif.org"
target="_blank">http://eubon-ipt.gbif.org</a> <<a
moz-do-not-send="true" href="http://eubon-ipt.gbif.org/"
target="_blank">http://eubon-ipt.gbif.org/</a>><br>
>><br>
>> [3]<br>
<a moz-do-not-send="true"
href="http://www.standardsingenomics.org/index.php/sigen/article/view/sigs.4898640"
target="_blank">http://www.standardsingenomics.org/index.php/sigen/article/view/sigs.4898640</a><br>
>><br>
>> [4] <<a moz-do-not-send="true"
href="http://links.gbif.org/sample_data_model" target="_blank">http://links.gbif.org/sample_data_model</a>><br>
<a moz-do-not-send="true"
href="http://links.gbif.org/sample_data_model" target="_blank">http://links.gbif.org/sample_data_model</a><br>
>><br>
>> [5] <a moz-do-not-send="true"
href="https://code.google.com/p/darwincore/issues/list" target="_blank">https://code.google.com/p/darwincore/issues/list</a><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> ____________________________________________________<br>
>><br>
>> ?amonn ? Tuama, M.Sc., Ph.D. (<a moz-do-not-send="true"
href="mailto:eotuama@gbif.org" target="_blank">eotuama@gbif.org</a>),<br>
>><br>
>> Senior Programme Officer for Interoperability,<br>
>><br>
>> Global Biodiversity Information Facility Secretariat,<br>
>><br>
>> Universitetsparken 15, DK-2100, Copenhagen ?, DENMARK<br>
>><br>
>> Phone: <a moz-do-not-send="true"
href="tel:%2B45%203532%201494" target="_blank">+45 3532 1494</a> <<a
moz-do-not-send="true" href="tel:%2B45%203532%201494">tel:%2B45%203532%201494</a>>
; Fax: <a moz-do-not-send="true" href="tel:%2B45%203532%201480"
target="_blank">+45 3532 1480</a><br>
<<a moz-do-not-send="true" href="tel:%2B45%203532%201480">tel:%2B45%203532%201480</a>><br>
>><br>
>><br>
>><br>
>><br>
>> _______________________________________________<br>
>> tdwg-content mailing list<br>
>> <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>> <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> _______________________________________________<br>
>> tdwg-content mailing list<br>
>> <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>><br>
>> <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>><br>
>><br>
>><br>
>> --<br>
>> Anne E. Thessen, Ph.D.<br>
>> The Data Detektiv, Owner and Founder<br>
>> Ronin Institute, Research Scholar<br>
>> <a moz-do-not-send="true" href="tel:443.225.9185"
target="_blank">443.225.9185</a><br>
>><br>
>><br>
>> _______________________________________________<br>
>> tdwg-content mailing list<br>
>> <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>> <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> --<br>
>> Anne E. Thessen, Ph.D.<br>
>> The Data Detektiv, Owner and Founder<br>
>> Ronin Institute, Research Scholar<br>
>> <a moz-do-not-send="true" href="tel:443.225.9185"
target="_blank">443.225.9185</a><br>
>><br>
>><br>
>><br>
>> _______________________________________________<br>
>> tdwg-content mailing list<br>
>> <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>> <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>><br>
>><br>
>><br>
>><br>
>> _______________________________________________<br>
>> tdwg-content mailing list<br>
>> <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>> <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>><br>
>><br>
>><br>
>> -------------- next part --------------<br>
>> An HTML attachment was scrubbed...<br>
>> URL:<br>
<a moz-do-not-send="true"
href="http://lists.tdwg.org/pipermail/tdwg-content/attachments/20140822/a46f067d/attachment-0001.html"
target="_blank">http://lists.tdwg.org/pipermail/tdwg-content/attachments/20140822/a46f067d/a<br>
ttachment-0001.html</a><br>
>><br>
>> ------------------------------<br>
>><br>
>> _______________________________________________<br>
>> tdwg-content mailing list<br>
>> <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>> <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
>><br>
>><br>
>> End of tdwg-content Digest, Vol 63, Issue 6<br>
>> *******************************************<br>
>><br>
>> _______________________________________________<br>
>> tdwg-content mailing list<br>
>> <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
>> <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> tdwg-content mailing list<br>
> <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
> <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> tdwg-content mailing list<br>
> <a moz-do-not-send="true"
href="mailto:tdwg-content@lists.tdwg.org" target="_blank">tdwg-content@lists.tdwg.org</a><br>
> <a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
<br>
<br>
<br>
_______________________________________________<br>
tdwg-content mailing list<br>
<a moz-do-not-send="true" href="mailto:tdwg-content@lists.tdwg.org"
target="_blank">tdwg-content@lists.tdwg.org</a><br>
<a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><br>
<br>
<br>
_______________________________________________<br>
tdwg-content mailing list<br>
<a moz-do-not-send="true" href="mailto:tdwg-content@lists.tdwg.org"
target="_blank">tdwg-content@lists.tdwg.org</a><br>
<a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><o:p></o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><span class="hoenzb"><span
style="color: rgb(136, 136, 136);">-- </span></span><span
style="color: rgb(136, 136, 136);"><br>
<span class="hoenzb">John Deck</span><br>
<span class="hoenzb"><a moz-do-not-send="true"
href="tel:%28541%29%20321-0689" target="_blank">(541) 321-0689</a></span></span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom: 12pt;"><br>
_______________________________________________<br>
tdwg-content mailing list<br>
<a moz-do-not-send="true" href="mailto:tdwg-content@lists.tdwg.org">tdwg-content@lists.tdwg.org</a><br>
<a moz-do-not-send="true"
href="http://lists.tdwg.org/mailman/listinfo/tdwg-content"
target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-content</a><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Steven J. Baskauf, Ph.D., Senior Lecturer
Vanderbilt University Dept. of Biological Sciences
postal mail address:
PMB 351634
Nashville, TN 37235-1634, U.S.A.
delivery address:
2125 Stevenson Center
1161 21st Ave., S.
Nashville, TN 37235
office: 2128 Stevenson Center
phone: (615) 343-4582, fax: (615) 322-4942
If you fax, please phone or email so that I will know to look for it.
<a class="moz-txt-link-freetext" href="http://bioimages.vanderbilt.edu">http://bioimages.vanderbilt.edu</a>
<a class="moz-txt-link-freetext" href="http://vanderbilt.edu/trees">http://vanderbilt.edu/trees</a>
</pre>
</body>
</html>