<html>
  <head>
    <meta content="text/html; charset=windows-1252"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    Ditto for iDigBio.<br>
    <br>
    - Alex<br>
    <br>
    <div class="moz-cite-prefix">On 08/24/2015 04:40 AM, John Wieczorek
      wrote:<br>
    </div>
    <blockquote
cite="mid:CAHwKGGdgOrZMzEVAMqAKkT3t1LoJt3gz8HbhMhOw0CQsc_bMwg@mail.gmail.com"
      type="cite">
      <meta http-equiv="Content-Type" content="text/html;
        charset=windows-1252">
      <div dir="ltr">I'd be very happy to work with you on that.</div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Mon, Aug 24, 2015 at 10:35 AM, Tim
          Robertson <span dir="ltr">&lt;<a moz-do-not-send="true"
              href="mailto:trobertson@gbif.org" target="_blank">trobertson@gbif.org</a>&gt;</span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">Id
            suggest TDWG hold back on this until the W3C CSV on the web
            group finish (Feb. 2016).<br>
            I submitted DwC-A as a use case which was accepted (<a
              moz-do-not-send="true"
              href="http://w3c.github.io/csvw/use-cases-and-requirements/"
              rel="noreferrer" target="_blank">http://w3c.github.io/csvw/use-cases-and-requirements/</a>)
            and have been following the progress.<br>
            <br>
            As far as I can tell the recommendations from that group
            will provide one possible future evolution of DwC-A covering
            tabular formats, encoding, micro syntax, JSON and RDF
            serialization and deserialisations, controlled terms,
            generic models (i.e. not star-schema) etc. It is because of
            this group that I have held back on updating the DwC text
            guidelines to address the issues we all know about, as I
            believe they will be covered there.<br>
            <br>
            By adopting W3C recommendations / standards, it will allow
            TDWG to focus on biodiversity specific issues - namely
            vocabularies and classes / models - and less on
            serialisation formats.<br>
            <br>
            I aim to write up / present a proposal on the future of
            DwC-A built around the recommendations to coincide with the
            conclusion of the W3C group. It should be a fairly logical
            progression from where we are today, and backwards
            compatibility looks doable. Id be very happy to work with
            others on that.<br>
            <br>
            Thanks,<br>
            Tim<br>
            <div class="HOEnZb">
              <div class="h5"><br>
                <br>
                On 21 Aug 2015, at 19:02, Alex Thompson &lt;<a
                  moz-do-not-send="true"
                  href="mailto:godfoder@acis.ufl.edu">godfoder@acis.ufl.edu</a>&gt;
                wrote:<br>
                <br>
                &gt; As someone who is normally a big proponent of JSON
                as a general<br>
                &gt; information representation format, I'd have to say
                that you're likely to<br>
                &gt; run into a myriad of issues with this. Chief among
                those would be that<br>
                &gt; JSON doesn't tend to play well with progressive
                decoding - most JSON<br>
                &gt; libraries force you to parse and decode the entire
                file, often in<br>
                &gt; memory, before you have access to any of the
                information. This works<br>
                &gt; fine for things like APIs where the information is
                generally quite<br>
                &gt; small, but for something a DwC-A, where each item
                can have 40-50<br>
                &gt; properties, and there can be hundreds of thousands
                of items, it quickly<br>
                &gt; gets un-manageable. There are progressive json
                decoding libraries in<br>
                &gt; most languages, but it is a hurdle to effective
                usage, since they<br>
                &gt; normally aren't part of the standard library
                packages.<br>
                &gt;<br>
                &gt; The two major strategies around this are either
                using some kind of<br>
                &gt; hybrid JSON-delimited (normally new line or null
                byte) format, or<br>
                &gt; writing hundreds of thousands of individual JSON
                files into a zip or tar<br>
                &gt; archive directly without first writing them to
                disk. I've tried both,<br>
                &gt; and I don't really like either of them for anything
                more than a quick hack.<br>
                &gt;<br>
                &gt; In terms of advanced serializations for DwC-A type
                data, I'd much rather<br>
                &gt; see something like DwC-SQLite, or DwC-HDF5 that
                would start to give us<br>
                &gt; some real tools to work with something other than a
                star schema.<br>
                &gt;<br>
                &gt; - Alex<br>
                &gt;<br>
                &gt; P.S.<br>
                &gt; You could always do this:<br>
                &gt;<br>
                &gt; meta.xml:<br>
                &gt; &lt;core encoding="utf-8" fieldsTerminatedBy="\t"
                linesTerminatedBy="\n"<br>
                &gt; fieldsEnclosedBy="" ignoreHeaderLines="1"<br>
                &gt; rowType="<a moz-do-not-send="true"
                  href="http://rs.tdwg.org/dwc/terms/Occurrence"
                  rel="noreferrer" target="_blank">http://rs.tdwg.org/dwc/terms/Occurrence</a>"&gt;<br>
                &gt;<br>
                &gt; occurrence.txt:<br>
                &gt; id,dynamicProperties<br>
                &gt; ABC123 &lt;tab&gt; {&lt;all of your actual
                data&gt;}<br>
                &gt;<br>
                &gt; On 08/21/2015 12:43 PM, Bob Morris wrote:<br>
                &gt;&gt; Is there or should there be a form of DwC-A
                serialized with JSON? If<br>
                &gt;&gt; no, should Interest Group X ( X= ???) initiate
                some discussion or<br>
                &gt;&gt; Task. If IG X is already at work, where is its
                discussion?<br>
                &gt;&gt;<br>
                &gt;&gt; Alternatively, should my question be something
                like "What is the JSON<br>
                &gt;&gt; alternative to DwC-A and where is it, or should
                it be, discussed?"<br>
                &gt;&gt;<br>
                &gt;&gt; Thanks<br>
                &gt;&gt; Bob<br>
                &gt;&gt;<br>
                &gt;<br>
                &gt; _______________________________________________<br>
                &gt; tdwg-tag mailing list<br>
                &gt; <a moz-do-not-send="true"
                  href="mailto:tdwg-tag@lists.tdwg.org">tdwg-tag@lists.tdwg.org</a><br>
                &gt; <a moz-do-not-send="true"
                  href="http://lists.tdwg.org/mailman/listinfo/tdwg-tag"
                  rel="noreferrer" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-tag</a><br>
                &gt;<br>
                <br>
                _______________________________________________<br>
                tdwg-tag mailing list<br>
                <a moz-do-not-send="true"
                  href="mailto:tdwg-tag@lists.tdwg.org">tdwg-tag@lists.tdwg.org</a><br>
                <a moz-do-not-send="true"
                  href="http://lists.tdwg.org/mailman/listinfo/tdwg-tag"
                  rel="noreferrer" target="_blank">http://lists.tdwg.org/mailman/listinfo/tdwg-tag</a><br>
              </div>
            </div>
          </blockquote>
        </div>
        <br>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
tdwg-tag mailing list
<a class="moz-txt-link-abbreviated" href="mailto:tdwg-tag@lists.tdwg.org">tdwg-tag@lists.tdwg.org</a>
<a class="moz-txt-link-freetext" href="http://lists.tdwg.org/mailman/listinfo/tdwg-tag">http://lists.tdwg.org/mailman/listinfo/tdwg-tag</a>
</pre>
    </blockquote>
    <br>
  </body>
</html>