I am almost done with the mapping of the dataset with the updated list of terms, but I need a bit more time because it’s very difficult to populate certain fields (e.g. eco:isLifeStageScopeComplete) due to it being at the dwc:Event level (limitation of star schema). I am also writing script to infer non-detection for presence-only data using Humboldt Extension for this dataset. This is still work in progress. I have what I need and logic in mind, but my R is rusty. I am doing this because it is otherwise difficult for me to see how I should populate the fields (and what not to do) in order to be able to interpret the data (make inference of non-detection).
With this exercise, I came to a conclusion that all of the Humboldt terms related to scope and completeness MUST be populated at the lowest level of dwc:Event in order to infer non-detection within a dataset. The results of the exercise and best practices, I believe, is secondary and not the most important for the ratification process, but I hope my conclusion above could help to shape the Hierarchical Events document.
Regarding the __ID fields that Debbie mentioned, I formatted the table with dwciri: prefix of the Humboldt terms “pretending” that this feature is available on IPT: https://github.com/gbif/ipt/issues/1947
This is especially useful for the scope terms (eco:targetTaxonomicScope, eco:targetLifeStageScope) This leads to my question - how to deal with list/array value for terms like dwciri:samplingPerformedBy? I noted that Ben Norton worked on a white paper about complex values/arrays (https://github.com/gbif/ipt/issues/1947#issuecomment-1447101388), wonder if anyone knows what happened?
Cheers
Ming
On 31 Jul 2023, at 16:52, Paul, Deborah <dlpaul@illinois.edu> wrote:Greetings John, et al.Stepping in gently here (having not been at all these conversations). So please feel free to ignore this if just too off-basae.I'm wondering why not addIdentification,identifiedByID? toIdentification,identifiedBy
Identification,identificationReferencesAnd coherently then,Sampling Effort,samplingPerformedByIDto go withSampling Effort,samplingPerformedBySame for Sampling and Vouchers?Material Collected,hasVouchers
Material Collected,voucherInstitutionsWho vouchered? (recordedBy, recordedByID?)And, for all of the above information outlined in the XML list of groups and related terms, where does the information go about who did this sampling, as part of what project? grant numbers? etc. The EML file? (Where's the metadata?). Thanks for clueing me in.Exciting to see the work getting to the place where it can be implemented 🙂
DebbieFrom: tdwg-humboldt <tdwg-humboldt-bounces@lists.tdwg.org> on behalf of John Wieczorek <tuco@berkeley.edu>
Sent: Monday, July 31, 2023 9:18 AM
To: Humboldt Core TG <tdwg-humboldt@lists.tdwg.org>
Cc: Wesley M. Hochachka <wmh6@cornell.edu>; tdwg-humboldt <tdwg-humboldt-bounces@lists.tdwg.org>
Subject: Re: [tdwg-humboldt] Humboldt next steps!_______________________________________________Hi folks,I hope the listserv is working.I have made the necessary edits to the eco: terms in preparation for a new extension XML file. I set up the script so that the extension XML can be generated from the term_versions.csv file (the canonical management file where the term versions are archived). The script allows the XML file to contain sections in which to group terms so that they can be seen and mapped together in the IPT. I have proposed some new group names and rearranged the terms in them in the hope that it might make the mapping process flow better in the IPT. The names of the group, the order of groups and the order of terms within the groups can all be set easily in a configuration file separate from the term_versions.csv file. It would be nice if we could decide as a Task Group on these groupings and orders before making a release. It will help people assess the proposal too, I believe. The groupings and term order I propose follow:group,term
Site,siteCount
Site,siteNestingDescription
Site,verbatimSiteDescriptions
Site,verbatimSiteNames
Site,geospatialScopeAreaInSquareKilometers
Site,totalAreaSampledInSquareKilometers
Site,reportedWeather
Site,reportedExtremeConditionsHabitat Scope,targetHabitatScope
Habitat Scope,excludedHabitatScopeTemporal Scope,eventDuration
Temporal Scope,eventDurationUnitTaxonomic Scope,targetTaxonomicScope
Taxonomic Scope,excludedTaxonomicScope
Taxonomic Scope,taxonCompletenessReported
Taxonomic Scope,taxonCompletenessProtocols
Taxonomic Scope,isTaxonomicScopeComplete
Taxonomic Scope,isAbsenceReported
Taxonomic Scope,absentTaxaOrganismal Scope,targetLifeStageScope
Organismal Scope,excludedLifeStageScope
Organismal Scope,isLifeStageScopeComplete
Organismal Scope,targetDegreeOfEstablishmentScope
Organismal Scope,excludedDegreeOfEstablishmentScope
Organismal Scope,isDegreeOfEstablishmentScopeComplete
Organismal Scope,targetGrowthFormScope
Organismal Scope,excludedGrowthFormScope
Organismal Scope,isGrowthFormScopeCompleteIdentification,identifiedBy
Identification,identificationReferencesMethodology Description,compilationType
Methodology Description,compilationSourceTypes
Methodology Description,inventoryTypes
Methodology Description,protocolNames
Methodology Description,protocolDescription
Methodology Description,protocolReferences
Methodology Description,isAbundanceReported
Methodology Description,isAbundanceCapReported
Methodology Description,abundanceCap
Methodology Description,isVegetationCoverReported
Methodology Description,isLeastSpecificTargetCategoryQuantityInclusiveMaterial Collected,hasVouchers
Material Collected,voucherInstitutions
Material Collected,hasMaterialSamples
Material Collected,materialSampleTypesSampling Effort,samplingPerformedBy
Sampling Effort,isSamplingEffortReported
Sampling Effort,samplingEffortProtocol
Sampling Effort,samplingEffortValue
Sampling Effort,samplingEffortUnitThe original groups and term order were:General Dataset & Identification,samplingPerformedBy
General Dataset & Identification,identificationReferences
General Dataset & Identification,identifiedByGeospatial Scope & Habitat,geospatialScopeAreaInSquareKilometers
Geospatial Scope & Habitat,totalAreaSampledInSquareKilometers
Geospatial Scope & Habitat,siteNestingDescription
Geospatial Scope & Habitat,siteCount
Geospatial Scope & Habitat,verbatimSiteDescriptions
Geospatial Scope & Habitat,verbatimSiteNamesGeospatial Scope & Habitat,targetHabitatScopeGeospatial Scope & Habitat,excludedHabitatScopeGeospatial Scope & Habitat,reportedWeatherGeospatial Scope & Habitat,reportedExtremeConditionsTemporal Scope,eventDuration
Temporal Scope,eventDurationUnitMethodology Description,compilationTypeTaxonomic & organismal Scope,targetTaxonomicScopeTaxonomic & organismal Scope,targetDegreeOfEstablishmentScopeTaxonomic & organismal Scope,targetGrowthFormScopeTaxonomic & organismal Scope,targetLifeStageScopeTaxonomic & organismal Scope,excludedTaxonomicScopeTaxonomic & organismal Scope,excludedDegreeOfEstablishmentScopeTaxonomic & organismal Scope,excludedGrowthFormScopeTaxonomic & organismal Scope,excludedLifeStageScope
Methodology Description,compilationSourceTypes
Methodology Description,inventoryTypes
Methodology Description,protocolNames
Methodology Description,protocolDescription
Methodology Description,protocolReferences
Methodology Description,isAbundanceReported
Methodology Description,isAbundanceCapReported
Methodology Description,abundanceCap
Methodology Description,isVegetationCoverReported
Methodology Description,hasVouchers
Methodology Description,voucherInstitutions
Methodology Description,hasMaterialSamples
Methodology Description,materialSampleTypes
Methodology Description,isLeastSpecificTargetCategoryQuantityInclusiveCompleteness & Effort,isSamplingEffortReportedCompleteness & Effort,samplingEffortProtocolCompleteness & Effort,samplingEffortValueCompleteness & Effort,samplingEffortUnitCompleteness & Effort,taxonCompletenessReportedCompleteness & Effort,taxonCompletenessProtocolsCompleteness & Effort,isTaxonomicScopeCompleteCompleteness & Effort,isDegreeOfEstablishmentScopeCompleteCompleteness & Effort,isGrowthFormScopeCompleteCompleteness & Effort,isLifeStageScopeCompleteCompleteness & Effort,isAbsenceReportedCompleteness & Effort,absentTaxaOn Wed, Jul 26, 2023 at 12:14 PM Yanina Sica <yanina.sica@gmail.com> wrote:Hi all,_______________________________________________Sorry about the listserve errors, duplicate emails, and event cancellations..the universe conspired against us today. Despite this, we managed to meet today and had a very good and productive meeting.We have resolved our latest doubts regarding the principles in the hierarchical doc and summarized our next steps:- John will finish some small edits to the list of terms and update the IPT- Ming will map again the BROKE-West campaign with the latest eco: terms and guiding principles- John and Ming will populate some good examples in the hierarchical document- Dmitry will try to track GBIF's attention (and funding?) to push the Humboldt User Guide to the next level and transform it into a nice format that allows easy updates.I cleaned and edited some of the previous documents and organized the Drive.
- Ratification-related documents are in this folder:
- Implementation Experience Report for Humboldt extension for ecological inventories
- Hierarchical Events in Humboldt Extension for ecological inventories
- isLeastSpecificTargetCategoryQuantityInclusive
- List of terms in Humboldt Extension for ecological inventories
- Humboldt Extension for ecological inventories User Guide
Next week, we will take a look at the examples from Ming and John and, if we all agree, we could formally submit the extension!We are getting there!Thank you all for your last efforts!All the best!PS: You should have received a calendar invite for next week. In case you did not, please let me know. You can also find the info below:Yani
tdwg-humboldt mailing list
tdwg-humboldt@lists.tdwg.org
https://lists.tdwg.org/mailman/listinfo/tdwg-humboldt
tdwg-humboldt mailing list
tdwg-humboldt@lists.tdwg.org
https://lists.tdwg.org/mailman/listinfo/tdwg-humboldt
