Dear TDWG Members and Friends,
Those of you who attended the conference in Perth last year will remember Annie Simpson's explanation of how hard it still is for a group like the Global Invasive Species Information Network (GISIN) to take TDWG's work and apply it directly to solve data integration issues in their community.
As I have previously suggested, I believe we should look closely at GISIN's requirements and make sure that those in a similar situation have all the tools and documentation required to start connecting data easily in ways that are compatible with TDWG standards. We have most of the pieces in place. We just need to organise and document them better.
Invasive species are of great significance to many countries. This means that addressing GISIN's requirements is directly relevant to a large number of TDWG participants. However their scenario is also a template for data sharing in many other biodiversity-related projects. They need to aggregate information from many sources to populate several inter-related data models (BioStatus, Occurrences, ProfileURLs, ImpactStatus, ManagementStatus and DispersalStatus). Several of these are very close to existing data models based on TDWG standards (for example, the Occurrence model can be considered to be extended Darwin Core). Others are examples of the kind of community-specific data which all biodiversity projects need to share.
TDWG's goal should be to ensure that GISIN and similar groups can follow a simple set of instructions (a "cook book") and use standard tools to build a network of this kind, and that they can do so in a way which ensures that related communities can also benefit from the data they share. For example, if GISIN members use the GISIN toolkit to share data, we ought to make sure that no further steps are required for GBIF, OBIS and others to integrate GISIN Occurrence data into their own indexes of species occurrences, or for EOL, ALA and others to integrate GISIN ImpactStatus, ManagementStatus and DispersalStatus data into their species profiles.
I am therefore writing this email as an appeal for TDWG members to step forward and help to complete the work we have started. I am setting a challenge for us to do whatever still needs to be done to our standards, tools and data sharing recommendations so that we can produce such a cook book by the end of 2009. I don't believe this is an unrealistic goal. We can start work now on identifying and resolving issues. We can plan activities at the 2009 TDWG conference in November to resolve any issues still outstanding at that point and to hold hackathon activities to fill any gaps in our tool set. This work also aligns well with the e-Biosphere conference's plan to develop a roadmap for biodiversity informatics for the next 10 years.
We have added a page to the TDWG Invasive Species wiki to seed discussion of what we need to do - http://wiki.tdwg.org/twiki/bin/view/InvasiveSpecies/GisinRequirements. Please take the time to look at this page (and at the information on GISIN's requirements at http://wiki.tdwg.org/twiki/bin/view/InvasiveSpecies/WebHome and http://www.niiss.org/cwis438/websites/GISINDirectory/Tech/ProtocolSpecificat...). Append comments and suggestions on what else we may need to do to the wiki, or simply reply to this message. In particular, if you are interested in being involved with the relevant TDWG Interest Groups in addressing some of the listed requirements, please contact me or the relevant Interest Group leaders.
I believe this is the most significant thing that TDWG can do right now to support the work of biodiversity informatics worldwide. Please consider making your own contribution to make it happen.
Best wishes,
Donald
[cid:736345722@24032009-2DA9]
Donald Hobern, Director, Atlas of Living Australia CSIRO Entomology, GPO Box 1700, Canberra, ACT 2601 Phone: (02) 62464352 Mobile: 0437990208 Email: Donald.Hobern@csiro.aumailto:Donald.Hobern@csiro.au Web: http://www.ala.org.au/
Dear Donald, TDWG Members and Friends,
GBIF supports your drive to use the GISIN communitys requirements as a test bed to demonstrate how the various TDWG standards can be applied to solve their informatics issues. Last January, we convened a three day workshop for the main Darwin Core (DwC) developers to enable them to make the final push before submitting their work to the TDWG standards ratification process. The proposed DwC standard, now submitted, provides a mechanism for extending its base capabilities by adding new terms to the DwC vocabulary. The basic technical solution for developing an extensible data encoding / transfer schema is thus in place for invasive species data.
In addition, GBIF has just released the first public version of the Integrated Publishing Toolkit (IPT) v1.0 (http://www.gbif.org/News/NEWS1238568696) which was demonstrated to great interest at the last TDWG Conference in Perth. The goal of the IPT is to remove constraints to the publishing and flow across networks of biodiversity data. It provides simple interfaces to transfer complete data stores efficiently in order to simplify the publishing process and reduce the latency between data publication and discoverability through the GBIF indexes. Accommodating three main data types - taxon primary occurrence data, taxonomic checklists and resource metadata - the IPT enables users to publish data residing in local databases, upload existing files, and access central services to make use of standardised controlled vocabularies. The IPT uses a star schema, i.e., a one-to-many schema in which a core record (expressed as DwC) can be linked to many additional descriptors. The DwC record in IPT does not need to be an occurrence. For checklist publishing we use the same DwC elements, but the record in that case represents a species name, not an occurrence, and any star schema extensions would be linking to species. For GISIN, those extensions can cover relevant areas such as management, impact, descriptions, images, related documents, distributions, dispersal, etc. The challenge for us is therefore to help the GISIN community to define extensions to DwC and to develop customized (thematic) portals that process and present the invasive species data according to their particular requirements, e.g. tracking species movements.
The IPT allows the sharing and use of specialist extensions amongst the IPT-user community. It can also act as an outreach tool for the GISIN community as it includes a web application that allows browsing and searching of published data, along with basic mapping services including Open Geospatial Consortium Web Map Service and Web Feature Service.
We support the idea of a cook book that helps communities use and, where required, extend standards to enable sharing of biodiversity data. GBIF, through its Training portfolio will be able to contribute documentation and training materials on many topics, e.g., the IPT, metadata, web services.
With regards,
Éamonn
_______________________________________________
Éamonn Ó Tuama, M.Sc., Ph.D. ( mailto:eotuama@gbif.org eotuama@gbif.org),
Senior Programme Officer, Inventory, Discovery, Access (IDA),
Global Biodiversity Information Facility Secretariat,
Universitetsparken 15, DK-2100 Copenhagen Ø, DENMARK Phone: +45 3532 1494; Fax: +45 3532 1480
From: tdwg-bounces@lists.tdwg.org [mailto:tdwg-bounces@lists.tdwg.org] On Behalf Of Donald.Hobern@csiro.au Sent: 25 March 2009 00:43 To: tdwg@lists.tdwg.org Subject: [tdwg] Consolidating TDWG work
Dear TDWG Members and Friends,
Those of you who attended the conference in Perth last year will remember Annie Simpson's explanation of how hard it still is for a group like the Global Invasive Species Information Network (GISIN) to take TDWG's work and apply it directly to solve data integration issues in their community.
As I have previously suggested, I believe we should look closely at GISIN's requirements and make sure that those in a similar situation have all the tools and documentation required to start connecting data easily in ways that are compatible with TDWG standards. We have most of the pieces in place. We just need to organise and document them better.
Invasive species are of great significance to many countries. This means that addressing GISIN's requirements is directly relevant to a large number of TDWG participants. However their scenario is also a template for data sharing in many other biodiversity-related projects. They need to aggregate information from many sources to populate several inter-related data models (BioStatus, Occurrences, ProfileURLs, ImpactStatus, ManagementStatus and DispersalStatus). Several of these are very close to existing data models based on TDWG standards (for example, the Occurrence model can be considered to be extended Darwin Core). Others are examples of the kind of community-specific data which all biodiversity projects need to share.
TDWG's goal should be to ensure that GISIN and similar groups can follow a simple set of instructions (a "cook book") and use standard tools to build a network of this kind, and that they can do so in a way which ensures that related communities can also benefit from the data they share. For example, if GISIN members use the GISIN toolkit to share data, we ought to make sure that no further steps are required for GBIF, OBIS and others to integrate GISIN Occurrence data into their own indexes of species occurrences, or for EOL, ALA and others to integrate GISIN ImpactStatus, ManagementStatus and DispersalStatus data into their species profiles.
I am therefore writing this email as an appeal for TDWG members to step forward and help to complete the work we have started. I am setting a challenge for us to do whatever still needs to be done to our standards, tools and data sharing recommendations so that we can produce such a cook book by the end of 2009. I don't believe this is an unrealistic goal. We can start work now on identifying and resolving issues. We can plan activities at the 2009 TDWG conference in November to resolve any issues still outstanding at that point and to hold hackathon activities to fill any gaps in our tool set. This work also aligns well with the e-Biosphere conference's plan to develop a roadmap for biodiversity informatics for the next 10 years.
We have added a page to the TDWG Invasive Species wiki to seed discussion of what we need to do - http://wiki.tdwg.org/twiki/bin/view/InvasiveSpecies/GisinRequirements. Please take the time to look at this page (and at the information on GISIN's requirements at http://wiki.tdwg.org/twiki/bin/view/InvasiveSpecies/WebHome and http://www.niiss.org/cwis438/websites/GISINDirectory/Tech/ProtocolSpecificat ion.php). Append comments and suggestions on what else we may need to do to the wiki, or simply reply to this message. In particular, if you are interested in being involved with the relevant TDWG Interest Groups in addressing some of the listed requirements, please contact me or the relevant Interest Group leaders.
I believe this is the most significant thing that TDWG can do right now to support the work of biodiversity informatics worldwide. Please consider making your own contribution to make it happen.
Best wishes,
Donald
Donald Hobern, Director, Atlas of Living Australia
CSIRO Entomology, GPO Box 1700, Canberra, ACT 2601
Phone: (02) 62464352 Mobile: 0437990208
Email: mailto:Donald.Hobern@csiro.au Donald.Hobern@csiro.au
Well that's a revelation ...
At the ECAT SSC meeting during GB8 in Oaxaca in 2004 I recall saying that in the absence of a publishing tool to mobilize checklist data we should use DwC and DIGIR and alias some of the fields if required. I further recall the lack of enthusiasm by the 'IT' people present. It looks like this has partly happened just 4.5 years later. But how do we keep all this simply mobilized data sychronized unless this publishing tool (sorry, I've not looked at it in detail ... yet) has some 'services' associated with it so that a consumer can see what's been added and what's changed?
Paul
________________________________
From: tdwg-bounces@lists.tdwg.org [mailto:tdwg-bounces@lists.tdwg.org] On Behalf Of Eamonn O Tuama (GBIF) Sent: 06 April 2009 14:18 To: tdwg@lists.tdwg.org Subject: Re: [tdwg] Consolidating TDWG work
Dear Donald, TDWG Members and Friends,
GBIF supports your drive to use the GISIN community's requirements as a test bed to demonstrate how the various TDWG standards can be applied to solve their informatics issues. Last January, we convened a three day workshop for the main Darwin Core (DwC) developers to enable them to make the final push before submitting their work to the TDWG standards ratification process. The proposed DwC standard, now submitted, provides a mechanism for extending its base capabilities by adding new terms to the DwC vocabulary. The basic technical solution for developing an extensible data encoding / transfer schema is thus in place for invasive species data.
In addition, GBIF has just released the first public version of the Integrated Publishing Toolkit (IPT) v1.0 (http://www.gbif.org/News/NEWS1238568696) which was demonstrated to great interest at the last TDWG Conference in Perth. The goal of the IPT is to remove constraints to the publishing and flow across networks of biodiversity data. It provides simple interfaces to transfer complete data stores efficiently in order to simplify the publishing process and reduce the latency between data publication and discoverability through the GBIF indexes. Accommodating three main data types - taxon primary occurrence data, taxonomic checklists and resource metadata - the IPT enables users to publish data residing in local databases, upload existing files, and access central services to make use of standardised controlled vocabularies. The IPT uses a "star schema", i.e., a one-to-many schema in which a core record (expressed as DwC) can be linked to many additional descriptors. The DwC record in IPT does not need to be an occurrence. For checklist publishing we use the same DwC elements, but the record in that case represents a species name, not an occurrence, and any star schema extensions would be linking to species. For GISIN, those extensions can cover relevant areas such as management, impact, descriptions, images, related documents, distributions, dispersal, etc. The challenge for us is therefore to help the GISIN community to define extensions to DwC and to develop customized (thematic) portals that process and present the invasive species data according to their particular requirements, e.g. tracking species movements.
The IPT allows the sharing and use of specialist extensions amongst the IPT-user community. It can also act as an outreach tool for the GISIN community as it includes a web application that allows browsing and searching of published data, along with basic mapping services including Open Geospatial Consortium Web Map Service and Web Feature Service.
We support the idea of a "cook book" that helps communities use and, where required, extend standards to enable sharing of biodiversity data. GBIF, through its Training portfolio will be able to contribute documentation and training materials on many topics, e.g., the IPT, metadata, web services.
With regards,
Éamonn
_______________________________________________
Éamonn Ó Tuama, M.Sc., Ph.D. (eotuama@gbif.org mailto:eotuama@gbif.org ),
Senior Programme Officer, Inventory, Discovery, Access (IDA),
Global Biodiversity Information Facility Secretariat,
Universitetsparken 15, DK-2100 Copenhagen Ø, DENMARK Phone: +45 3532 1494; Fax: +45 3532 1480
From: tdwg-bounces@lists.tdwg.org [mailto:tdwg-bounces@lists.tdwg.org] On Behalf Of Donald.Hobern@csiro.au Sent: 25 March 2009 00:43 To: tdwg@lists.tdwg.org Subject: [tdwg] Consolidating TDWG work
Dear TDWG Members and Friends,
Those of you who attended the conference in Perth last year will remember Annie Simpson's explanation of how hard it still is for a group like the Global Invasive Species Information Network (GISIN) to take TDWG's work and apply it directly to solve data integration issues in their community.
As I have previously suggested, I believe we should look closely at GISIN's requirements and make sure that those in a similar situation have all the tools and documentation required to start connecting data easily in ways that are compatible with TDWG standards. We have most of the pieces in place. We just need to organise and document them better.
Invasive species are of great significance to many countries. This means that addressing GISIN's requirements is directly relevant to a large number of TDWG participants. However their scenario is also a template for data sharing in many other biodiversity-related projects. They need to aggregate information from many sources to populate several inter-related data models (BioStatus, Occurrences, ProfileURLs, ImpactStatus, ManagementStatus and DispersalStatus). Several of these are very close to existing data models based on TDWG standards (for example, the Occurrence model can be considered to be extended Darwin Core). Others are examples of the kind of community-specific data which all biodiversity projects need to share.
TDWG's goal should be to ensure that GISIN and similar groups can follow a simple set of instructions (a "cook book") and use standard tools to build a network of this kind, and that they can do so in a way which ensures that related communities can also benefit from the data they share. For example, if GISIN members use the GISIN toolkit to share data, we ought to make sure that no further steps are required for GBIF, OBIS and others to integrate GISIN Occurrence data into their own indexes of species occurrences, or for EOL, ALA and others to integrate GISIN ImpactStatus, ManagementStatus and DispersalStatus data into their species profiles.
I am therefore writing this email as an appeal for TDWG members to step forward and help to complete the work we have started. I am setting a challenge for us to do whatever still needs to be done to our standards, tools and data sharing recommendations so that we can produce such a cook book by the end of 2009. I don't believe this is an unrealistic goal. We can start work now on identifying and resolving issues. We can plan activities at the 2009 TDWG conference in November to resolve any issues still outstanding at that point and to hold hackathon activities to fill any gaps in our tool set. This work also aligns well with the e-Biosphere conference's plan to develop a roadmap for biodiversity informatics for the next 10 years.
We have added a page to the TDWG Invasive Species wiki to seed discussion of what we need to do - http://wiki.tdwg.org/twiki/bin/view/InvasiveSpecies/GisinRequirements. Please take the time to look at this page (and at the information on GISIN's requirements at http://wiki.tdwg.org/twiki/bin/view/InvasiveSpecies/WebHome and http://www.niiss.org/cwis438/websites/GISINDirectory/Tech/ProtocolSpecificat...). Append comments and suggestions on what else we may need to do to the wiki, or simply reply to this message. In particular, if you are interested in being involved with the relevant TDWG Interest Groups in addressing some of the listed requirements, please contact me or the relevant Interest Group leaders.
I believe this is the most significant thing that TDWG can do right now to support the work of biodiversity informatics worldwide. Please consider making your own contribution to make it happen.
Best wishes,
Donald
Donald Hobern, Director, Atlas of Living Australia
CSIRO Entomology, GPO Box 1700, Canberra, ACT 2601
Phone: (02) 62464352 Mobile: 0437990208
Email: Donald.Hobern@csiro.au mailto:Donald.Hobern@csiro.au
Find out about CABI's global summit on 'Food security in a climate of change' at www.cabiglobalsummit.com 19 - 21 October 2009, London, UK.
************************************************************************ The information contained in this e-mail and any files transmitted with it is confidential and is for the exclusive use of the intended recipient. If you are not the intended recipient please note that any distribution, copying or use of this communication or the information in it is prohibited.
Whilst CAB International trading as CABI takes steps to prevent the transmission of viruses via e-mail, we cannot guarantee that any e-mail or attachment is free from computer viruses and you are strongly advised to undertake your own anti-virus precautions.
If you have received this communication in error, please notify us by e-mail at cabi@cabi.org or by telephone on +44 (0)1491 829199 and then delete the e-mail and any copies of it.
CABI is an International Organization recognised by the UK Government under Statutory Instrument 1982 No. 1071.
**************************************************************************
At the ECAT SSC meeting during GB8 in Oaxaca in 2004 I recall saying that in the absence of a publishing tool to mobilize checklist data we should use DwC and DIGIR and alias some of the fields if required. I further recall the lack of enthusiasm by the 'IT' people present. It looks like this has partly happened just 4.5 years later. But how do we keep all this simply mobilized data sychronized unless this publishing tool (sorry, I've not looked at it in detail ... yet) has some 'services' associated with it so that a consumer can see what's been added and what's changed?
Paul
good question, Paul.
The IPT does have an embedded data cache and with every new version published it is first uploaded into the cache. At that moment potentially previously existing records, before being removed for performance reasons, are compared to new ones and it is thereby detected which records have changed, which were deleted and which are new. This information is in IPTs first version only accessible via the full dataset dump (aka darwin core archive), but it is planned to expose this also as a pageable atom feed and if requested by serious users, potentially also as OAI-PMH.
Markus
participants (4)
-
"Markus Döring (GBIF)" -
Donald.Hobern＠csiro.au -
Eamonn O Tuama (GBIF) -
Paul Kirk