- tdwg - lists.tdwg.org

Re: Standards for date / time values?
by John R. WIECZOREK 24 Feb '06

24 Feb '06

I concur with Richard on the sufficiency of the three fields, and prefer to err on the side of simplicity for the DwC. On 2/24/06, Richard Pyle <deepreef(a)bishopmuseum.org> wrote: > > RE: CollectingDatesInterpreted > When would this field *not* be set to true? In other words, what is the > threshold for "interpreted", in terms of populating EarliestCollectingDate > and LatestCollectingDate? > > At one extreme, we have a VerbatimDateTime of "12 January 2005, 13:34:15 > local time". Not a lot of interpretation needed there (other than > conversion of original documentation format to ASCII/Unicode, and > interpretation of the time zone from the locality data). But conversion > of > "12 January 2005" to > > EarliestCollectingDate: "2005-01-12 00:00:00" > LatestCollectingDate: "2005-01-12 23:59:59" > > ...requires a bit of interpretation. Maybe the time portion could simply > be > omitted; but even still, there is technically *some* interpretation when > converting "January" to "01". > > And the spectrum continues on from there up until, well, stuff that would > be > unambiguously flagged as "interpreted". > > My point is, there should be some sort of guideline as to when to set > CollectingDatesInterpreted true; i.e., what threshold of interpretation > triggers that field. If such a guideline already exists, then I apologize > for missing it. > > Personally, I don't think CollectingDatesInterpreted is necessary, really, > because I have always regarded my equivalents of EarliestCollectingDate > and > LatestCollectingDate as interpreted. > > RE: Lynn's examples below. > I wrestled with these issues (and more) a number of years ago. Other > examples include "Summer; year unknown" (important for seasonal species), > and multiple (non-contiguous) date ranges: "May or August, 2004" -- among > others. > > In addition to the equivalents of VerbatimDateTime, > EarliestCollectingDate, > LatestCollectingDate; I had a controlled-vocabulary field called > "DateRangeQualifier", which had an enumerated list of options tied to > interpretation logic (e.g., "continuous range", "discontinuous range", > "unknown point", "either/or", etc.; as well as sub-range qualifiers like > "Summer"). I modelled it after an analogous field I use for interpreting > paired or multiple georef coordinates (e.g., "transect" vs. "bounding > box"; > "polygon" vs. "linear"). This DateRangeQualifier approach ended up being > too > cumbersome to implement -- only because I was too lazy, and the number of > cases that benefited from it (i.e., allowed computer logic to be applied, > independent of a human reading of the VerbatimDateTime value) was low. > > The three fields > VerbatimDateTime/EarliestCollectingDate/LatestCollectingDate will give you > 90% of the value 90% of the time (at least for the specimen/observation > data > that I deal with). I wonder if DWC really needs more precision capability > than that.... > > Rich > > Richard L. Pyle, PhD > Database Coordinator for Natural Sciences > Department of Natural Sciences, Bishop Museum > 1525 Bernice St., Honolulu, HI 96817 > Ph: (808)848-4115, Fax: (808)847-8252 > email: deepreef(a)bishopmuseum.org > http://hbs.bishopmuseum.org/staff/pylerichard.html > > > > -----Original Message----- > From: Taxonomic Databases Working Group List > [mailto:TDWG@LISTSERV.NHM.KU.EDU]On Behalf Of Lynn Kutner > Sent: Thursday, February 23, 2006 5:26 PM > To: TDWG(a)LISTSERV.NHM.KU.EDU > Subject: Re: Standards for date / time values? > > > Hi Stan and all - > > Yes, I am thinking about dates in the context of the observation / > monitoring schema. > > Here are some of the types of dates that I've seen in just the data from > the > NatureServe member programs based on either lack of available information > or > interesting data collection situations. > > 1) partial dates (only have the year or year-month) - definitely > accommodated by the ISO 8601 format > > 2) vague dates - for these I suppose you'd use ISO 8601 in > EarliestCollectingDate / LatestCollectingDate, VerbatimDateTime, and the > CollectingDatesInterpreted flag ? > > PRE-1975 > post-1992 > summer 2001 > Mid-1990s > late 1950's > Circa 1941 > Early 1990s > > 3) a range of dates indicating that collection occurred on a specific date > sometime between those two limits and you're not sure when (example: > Observations / collections from pitfall traps may have date ranges > associated with them rather than a single date. For instance, you can have > dates of collection of 19Dec 2004 to 23Jan 2005 and the specimen was from > some unknown specific date in that rage), > > 4) continuous data collection over a period of time (some sort of sensor) > > 5) information collected on one of two possible dates (one or the other is > correct, it's not a range) (example: aerial surveys flown on either May 27 > or June 04, 1999). > > It seems that EarliestCollectingDate / LatestCollectingDate could be used > for #3, #4, and #5 - but is there a way to differentiate between these as > actually being very different types of dates? Would those distinctions > just > be communicated using the VerbatimDateTime? > > Of course for all of these examples the fallback would be to use the > VerbatimDateTime, but it would be ideal to have as much as possible of the > date information in field(s) that can be queried and analyzed > > Thank you for your thoughts and help. > > Lynn > > > > Lynn Kutner > NatureServe > www.natureserve.org > (303) 541-0360 > lynn_kutner(a)natureserve.org > > > > From: Blum, Stan [mailto:sblum@calacademy.org] > Sent: Thu 2/23/2006 6:41 PM > To: Lynn Kutner; TDWG(a)LISTSERV.NHM.KU.EDU > Subject: RE: Standards for date / time values? > > > Hi Lynn, > > Context is everything, so I'm going to assume you are talking about > date/time representations in the observation and monitoring schema(s) that > you are developing. The thinking in the collections community has gone > something along the lines of this: > > Concept: the date-time of a collecting event [= recording-event, > gathering-event] -- i.e., when one or more organisms were collected or > observed -- expressed in the common Gregorian calendar, in local time. > > Requirements: > > 1 - express the data/time exactly as it was recorded > 2 - retreive records by date/time ranges (date-time greater than X and/or > less than Y) > 3 - accommodate date-times of varying precision, including explicit > date-time ranges (specifying a duration) > 4 - support seasonal (time of year) queries > > These requirements can be met by the following fields: > > VerbatimDateTime > EarliestDateCollected > LatestDateCollected > CollectingDatesInterpreted > DayOfYear > > As John Wieczorek noted, the ISO 8601 format accommodates varying degrees > of > precision. Using a "verbatim date-time" [text string] is as good as you > can > do to satisfy the first requirement, short of scanning field notes or > voice > recordings. The earliest and latest dates support the recording of > explicit > ranges as well as interpreted ranges, which could be used to make "Summer > of > 1952" retrievable. The CollectingDatesInterpreted field would be a > boolean > field, set as true when the earliest and latest dates represent > interpretations rather than an explicit range. The "DayOfYear" field is a > compromise that we've offered as a simple way to support queries involving > seasonal (annual) cycles; e.g., collected in the northern summer, > regardless > of year. But it can be argued that day of year is derivable from the > other > fields (unless year is unknown), and that it doesn't accommodate explicit > ranges. > > A bit more documentation is needed to address the odd cases (What do I do > when ...?), but these five fields will support a lot of the data exchange > needed for this concept. These fields are not intended to handle the > dating > of localities in paleontology, nor are they intended to handle named > periods > that are used in cultural collections (e.g., Iron Age, Victorian). > > ABCD uses a few more fields to handle the concept, > http://ww3.bgbm.org/abcddocs/AbcdConcepts but some of these (date, time, > and time zone) are handled by the ISO format, except when the larger units > are unkown; .e.g., the year is unknown, but the day is [ June 16 ]; or the > date is unknown, but the time is [ 14:00-15:30 ]. > > AbcdConcept0860 /DataSets/DataSet/Units/Unit/Gathering/DateTime > AbcdConcept0861 /DataSets/DataSet/Units/Unit/Gathering/DateTime/DateText > AbcdConcept0862 /DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeZone > AbcdConcept0863 > /DataSets/DataSet/Units/Unit/Gathering/DateTime/ISODateTimeBegin > AbcdConcept0864 > /DataSets/DataSet/Units/Unit/Gathering/DateTime/DayNumberBegin > AbcdConcept0865 > /DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeOfDayBegin > AbcdConcept0866 > /DataSets/DataSet/Units/Unit/Gathering/DateTime/ISODateTimeEnd > AbcdConcept0867 > /DataSets/DataSet/Units/Unit/Gathering/DateTime/DayNumberEnd > AbcdConcept0868 > /DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeOfDayEnd > AbcdConcept0869 > /DataSets/DataSet/Units/Unit/Gathering/DateTime/PeriodExplicit > The ABCD fields pretty much cover the entire concept space. One could > argue > whether time-zone is relevant to the description of biological phenomena, > but we do know that ship trawls do cross time-zones (including the > date-line), and that daylight savings time could stretch or compress some > nocturnal collecting events if their durations were calculated too simply. > > To some extent these arguments are still going on, so analyze your > requirements and your data, then state your position. ;-) > > Cheers, > > -Stan > Stanley D. Blum, Ph.D. > Research Information Manager > California Academy of Sciences > 875 Howard St. > San Francisco, CA > +1 (415) 321-8183 > > > -----Original Message----- > From: Taxonomic Databases Working Group List > [mailto:TDWG@LISTSERV.NHM.KU.EDU] On Behalf Of Lynn Kutner > Sent: Thursday, February 23, 2006 9:12 AM > To: TDWG(a)LISTSERV.NHM.KU.EDU > Subject: Standards for date / time values? > > > Hi - > I'm working with a suite of date attributes that can include a combination > of precise dates, imprecise dates, and ranges of dates (and the same types > of time values). We'd like to follow existing standards. If this sort of > date / time standard exists, I'd appreciate leads to the appropriate > resources. > Thank you for your help - > Lynn > > > Lynn Kutner > Data Management Coordinator > NatureServe > Email: lynn_kutner(a)natureserve.org > Phone: (303) 541-0360 > www.natureserve.org >

1 0

Re: Standards for date / time values?
by Patricia Koleff Osorio 24 Feb '06

24 Feb '06

We should be clear on format for Early and Latest CollectionDate, meaning to store data as DDMMYYYY There are some old specimens missing the day or even the month (and only have years, and even not the complete year, only the like 79) I think that time hour should be another field, noting that must be refered to the locality Regards Patricia Koleff At 12:00 a.m. 24/02/2006 -1000, Richard Pyle wrote: >RE: CollectingDatesInterpreted >When would this field *not* be set to true? In other words, what is the >threshold for "interpreted", in terms of populating EarliestCollectingDate >and LatestCollectingDate? > >At one extreme, we have a VerbatimDateTime of "12 January 2005, 13:34:15 >local time". Not a lot of interpretation needed there (other than >conversion of original documentation format to ASCII/Unicode, and >interpretation of the time zone from the locality data). But conversion of >"12 January 2005" to > >EarliestCollectingDate: "2005-01-12 00:00:00" >LatestCollectingDate: "2005-01-12 23:59:59" > >...requires a bit of interpretation. Maybe the time portion could simply be >omitted; but even still, there is technically *some* interpretation when >converting "January" to "01". > >And the spectrum continues on from there up until, well, stuff that would be >unambiguously flagged as "interpreted". > >My point is, there should be some sort of guideline as to when to set >CollectingDatesInterpreted true; i.e., what threshold of interpretation >triggers that field. If such a guideline already exists, then I apologize >for missing it. > >Personally, I don't think CollectingDatesInterpreted is necessary, really, >because I have always regarded my equivalents of EarliestCollectingDate and >LatestCollectingDate as interpreted. > >RE: Lynn's examples below. >I wrestled with these issues (and more) a number of years ago. Other >examples include "Summer; year unknown" (important for seasonal species), >and multiple (non-contiguous) date ranges: "May or August, 2004" -- among >others. > >In addition to the equivalents of VerbatimDateTime, EarliestCollectingDate, >LatestCollectingDate; I had a controlled-vocabulary field called >"DateRangeQualifier", which had an enumerated list of options tied to >interpretation logic (e.g., "continuous range", "discontinuous range", >"unknown point", "either/or", etc.; as well as sub-range qualifiers like >"Summer"). I modelled it after an analogous field I use for interpreting >paired or multiple georef coordinates (e.g., "transect" vs. "bounding box"; >"polygon" vs. "linear"). This DateRangeQualifier approach ended up being too >cumbersome to implement -- only because I was too lazy, and the number of >cases that benefited from it (i.e., allowed computer logic to be applied, >independent of a human reading of the VerbatimDateTime value) was low. > >The three fields >VerbatimDateTime/EarliestCollectingDate/LatestCollectingDate will give you >90% of the value 90% of the time (at least for the specimen/observation data >that I deal with). I wonder if DWC really needs more precision capability >than that.... > >Rich > >Richard L. Pyle, PhD >Database Coordinator for Natural Sciences >Department of Natural Sciences, Bishop Museum >1525 Bernice St., Honolulu, HI 96817 >Ph: (808)848-4115, Fax: (808)847-8252 >email: deepreef(a)bishopmuseum.org >http://hbs.bishopmuseum.org/staff/pylerichard.html > > > >-----Original Message----- >From: Taxonomic Databases Working Group List >[mailto:TDWG@LISTSERV.NHM.KU.EDU]On Behalf Of Lynn Kutner >Sent: Thursday, February 23, 2006 5:26 PM >To: TDWG(a)LISTSERV.NHM.KU.EDU >Subject: Re: Standards for date / time values? > > >Hi Stan and all - > >Yes, I am thinking about dates in the context of the observation / >monitoring schema. > >Here are some of the types of dates that I've seen in just the data from the >NatureServe member programs based on either lack of available information or >interesting data collection situations. > >1) partial dates (only have the year or year-month) - definitely >accommodated by the ISO 8601 format > >2) vague dates - for these I suppose you'd use ISO 8601 in >EarliestCollectingDate / LatestCollectingDate, VerbatimDateTime, and the >CollectingDatesInterpreted flag ? > >PRE-1975 >post-1992 >summer 2001 >Mid-1990s >late 1950's >Circa 1941 >Early 1990s > >3) a range of dates indicating that collection occurred on a specific date >sometime between those two limits and you're not sure when (example: >Observations / collections from pitfall traps may have date ranges >associated with them rather than a single date. For instance, you can have >dates of collection of 19Dec 2004 to 23Jan 2005 and the specimen was from >some unknown specific date in that rage), > >4) continuous data collection over a period of time (some sort of sensor) > >5) information collected on one of two possible dates (one or the other is >correct, it's not a range) (example: aerial surveys flown on either May 27 >or June 04, 1999). > >It seems that EarliestCollectingDate / LatestCollectingDate could be used >for #3, #4, and #5 - but is there a way to differentiate between these as >actually being very different types of dates? Would those distinctions just >be communicated using the VerbatimDateTime? > >Of course for all of these examples the fallback would be to use the >VerbatimDateTime, but it would be ideal to have as much as possible of the >date information in field(s) that can be queried and analyzed > >Thank you for your thoughts and help. > >Lynn > > > >Lynn Kutner >NatureServe >www.natureserve.org >(303) 541-0360 >lynn_kutner(a)natureserve.org > > > >From: Blum, Stan [mailto:sblum@calacademy.org] >Sent: Thu 2/23/2006 6:41 PM >To: Lynn Kutner; TDWG(a)LISTSERV.NHM.KU.EDU >Subject: RE: Standards for date / time values? > > >Hi Lynn, > >Context is everything, so I'm going to assume you are talking about >date/time representations in the observation and monitoring schema(s) that >you are developing. The thinking in the collections community has gone >something along the lines of this: > >Concept: the date-time of a collecting event [= recording-event, >gathering-event] -- i.e., when one or more organisms were collected or >observed -- expressed in the common Gregorian calendar, in local time. > >Requirements: > >1 - express the data/time exactly as it was recorded >2 - retreive records by date/time ranges (date-time greater than X and/or >less than Y) >3 - accommodate date-times of varying precision, including explicit >date-time ranges (specifying a duration) >4 - support seasonal (time of year) queries > >These requirements can be met by the following fields: > >VerbatimDateTime >EarliestDateCollected >LatestDateCollected >CollectingDatesInterpreted >DayOfYear > >As John Wieczorek noted, the ISO 8601 format accommodates varying degrees of >precision. Using a "verbatim date-time" [text string] is as good as you can >do to satisfy the first requirement, short of scanning field notes or voice >recordings. The earliest and latest dates support the recording of explicit >ranges as well as interpreted ranges, which could be used to make "Summer of >1952" retrievable. The CollectingDatesInterpreted field would be a boolean >field, set as true when the earliest and latest dates represent >interpretations rather than an explicit range. The "DayOfYear" field is a >compromise that we've offered as a simple way to support queries involving >seasonal (annual) cycles; e.g., collected in the northern summer, regardless >of year. But it can be argued that day of year is derivable from the other >fields (unless year is unknown), and that it doesn't accommodate explicit >ranges. > >A bit more documentation is needed to address the odd cases (What do I do >when ...?), but these five fields will support a lot of the data exchange >needed for this concept. These fields are not intended to handle the dating >of localities in paleontology, nor are they intended to handle named periods >that are used in cultural collections (e.g., Iron Age, Victorian). > >ABCD uses a few more fields to handle the concept, >http://ww3.bgbm.org/abcddocs/AbcdConcepts but some of these (date, time, >and time zone) are handled by the ISO format, except when the larger units >are unkown; .e.g., the year is unknown, but the day is [ June 16 ]; or the >date is unknown, but the time is [ 14:00-15:30 ]. > >AbcdConcept0860 /DataSets/DataSet/Units/Unit/Gathering/DateTime >AbcdConcept0861 /DataSets/DataSet/Units/Unit/Gathering/DateTime/DateText >AbcdConcept0862 /DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeZone >AbcdConcept0863 >/DataSets/DataSet/Units/Unit/Gathering/DateTime/ISODateTimeBegin >AbcdConcept0864 >/DataSets/DataSet/Units/Unit/Gathering/DateTime/DayNumberBegin >AbcdConcept0865 >/DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeOfDayBegin >AbcdConcept0866 >/DataSets/DataSet/Units/Unit/Gathering/DateTime/ISODateTimeEnd >AbcdConcept0867 >/DataSets/DataSet/Units/Unit/Gathering/DateTime/DayNumberEnd >AbcdConcept0868 >/DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeOfDayEnd >AbcdConcept0869 >/DataSets/DataSet/Units/Unit/Gathering/DateTime/PeriodExplicit >The ABCD fields pretty much cover the entire concept space. One could argue >whether time-zone is relevant to the description of biological phenomena, >but we do know that ship trawls do cross time-zones (including the >date-line), and that daylight savings time could stretch or compress some >nocturnal collecting events if their durations were calculated too simply. > >To some extent these arguments are still going on, so analyze your >requirements and your data, then state your position. ;-) > >Cheers, > >-Stan >Stanley D. Blum, Ph.D. >Research Information Manager >California Academy of Sciences >875 Howard St. >San Francisco, CA >+1 (415) 321-8183 > > >-----Original Message----- >From: Taxonomic Databases Working Group List >[mailto:TDWG@LISTSERV.NHM.KU.EDU] On Behalf Of Lynn Kutner >Sent: Thursday, February 23, 2006 9:12 AM >To: TDWG(a)LISTSERV.NHM.KU.EDU >Subject: Standards for date / time values? > > >Hi - >I'm working with a suite of date attributes that can include a combination >of precise dates, imprecise dates, and ranges of dates (and the same types >of time values). We'd like to follow existing standards. If this sort of >date / time standard exists, I'd appreciate leads to the appropriate >resources. >Thank you for your help - >Lynn > > >Lynn Kutner >Data Management Coordinator >NatureServe >Email: lynn_kutner(a)natureserve.org >Phone: (303) 541-0360 >www.natureserve.org Dra. Patricia Koleff Directora de Análisis y Prioridades CONABIO Comisión Nacional para el Conocimiento y Uso de la Biodiversidad Av. Liga Periferico - Insurgentes Sur 4903 Col. Parques del Pedregal Delegacion Tlalpan 14010 Mexico, D.F. MEXICO Tel. +52 (55) 5528 9105 Fax +52 (55) 5528 9194 www.conabio.gob.mx e-mail: pkoleff(a)xolo.conabio.gob.mx

1 0

Re: Standards for date / time values?
by John R. WIECZOREK 24 Feb '06

24 Feb '06

Patricia, and all, The ISO 8601 standard specifies the format explicitly. The W3C implementation of it is explained at http://www.w3.org/TR/NOTE-datetime. Your point is well taken though, some interpretation of original data (e.g., expanding two-digit years to four digits) will often have to be done to express dates as xsd:dateTime. Can you justify the separation (or addition) of the time fields? It may ease the expression of queries on time independent of date. It may also allow one to express a discrete across a date range (e.g., 17:00 every day in June 2005). This seems like an exceptional case. Is that worth adding two more concepts to accomodate this? John On 2/24/06, Patricia Koleff Osorio <pkoleff(a)xolo.conabio.gob.mx> wrote: > > We should be clear on format for Early and Latest CollectionDate, meaning > to store data as DDMMYYYY > There are some old specimens missing the day or even the month (and only > have years, and even not the complete year, only the like 79) > I think that time hour should be another field, noting that must be > refered > to the locality > Regards > Patricia Koleff > > > At 12:00 a.m. 24/02/2006 -1000, Richard Pyle wrote: > >RE: CollectingDatesInterpreted > >When would this field *not* be set to true? In other words, what is the > >threshold for "interpreted", in terms of populating > EarliestCollectingDate > >and LatestCollectingDate? > > > >At one extreme, we have a VerbatimDateTime of "12 January 2005, 13:34:15 > >local time". Not a lot of interpretation needed there (other than > >conversion of original documentation format to ASCII/Unicode, and > >interpretation of the time zone from the locality data). But conversion > of > >"12 January 2005" to > > > >EarliestCollectingDate: "2005-01-12 00:00:00" > >LatestCollectingDate: "2005-01-12 23:59:59" > > > >...requires a bit of interpretation. Maybe the time portion could simply > be > >omitted; but even still, there is technically *some* interpretation when > >converting "January" to "01". > > > >And the spectrum continues on from there up until, well, stuff that would > be > >unambiguously flagged as "interpreted". > > > >My point is, there should be some sort of guideline as to when to set > >CollectingDatesInterpreted true; i.e., what threshold of interpretation > >triggers that field. If such a guideline already exists, then I apologize > >for missing it. > > > >Personally, I don't think CollectingDatesInterpreted is necessary, > really, > >because I have always regarded my equivalents of EarliestCollectingDate > and > >LatestCollectingDate as interpreted. > > > >RE: Lynn's examples below. > >I wrestled with these issues (and more) a number of years ago. Other > >examples include "Summer; year unknown" (important for seasonal species), > >and multiple (non-contiguous) date ranges: "May or August, 2004" -- among > >others. > > > >In addition to the equivalents of VerbatimDateTime, > EarliestCollectingDate, > >LatestCollectingDate; I had a controlled-vocabulary field called > >"DateRangeQualifier", which had an enumerated list of options tied to > >interpretation logic (e.g., "continuous range", "discontinuous range", > >"unknown point", "either/or", etc.; as well as sub-range qualifiers like > >"Summer"). I modelled it after an analogous field I use for interpreting > >paired or multiple georef coordinates (e.g., "transect" vs. "bounding > box"; > >"polygon" vs. "linear"). This DateRangeQualifier approach ended up being > too > >cumbersome to implement -- only because I was too lazy, and the number of > >cases that benefited from it (i.e., allowed computer logic to be applied, > >independent of a human reading of the VerbatimDateTime value) was low. > > > >The three fields > >VerbatimDateTime/EarliestCollectingDate/LatestCollectingDate will give > you > >90% of the value 90% of the time (at least for the specimen/observation > data > >that I deal with). I wonder if DWC really needs more precision capability > >than that.... > > > >Rich > > > >Richard L. Pyle, PhD > >Database Coordinator for Natural Sciences > >Department of Natural Sciences, Bishop Museum > >1525 Bernice St., Honolulu, HI 96817 > >Ph: (808)848-4115, Fax: (808)847-8252 > >email: deepreef(a)bishopmuseum.org > >http://hbs.bishopmuseum.org/staff/pylerichard.html > > > > > > > >-----Original Message----- > >From: Taxonomic Databases Working Group List > >[mailto:TDWG@LISTSERV.NHM.KU.EDU]On Behalf Of Lynn Kutner > >Sent: Thursday, February 23, 2006 5:26 PM > >To: TDWG(a)LISTSERV.NHM.KU.EDU > >Subject: Re: Standards for date / time values? > > > > > >Hi Stan and all - > > > >Yes, I am thinking about dates in the context of the observation / > >monitoring schema. > > > >Here are some of the types of dates that I've seen in just the data from > the > >NatureServe member programs based on either lack of available information > or > >interesting data collection situations. > > > >1) partial dates (only have the year or year-month) - definitely > >accommodated by the ISO 8601 format > > > >2) vague dates - for these I suppose you'd use ISO 8601 in > >EarliestCollectingDate / LatestCollectingDate, VerbatimDateTime, and the > >CollectingDatesInterpreted flag ? > > > >PRE-1975 > >post-1992 > >summer 2001 > >Mid-1990s > >late 1950's > >Circa 1941 > >Early 1990s > > > >3) a range of dates indicating that collection occurred on a specific > date > >sometime between those two limits and you're not sure when (example: > >Observations / collections from pitfall traps may have date ranges > >associated with them rather than a single date. For instance, you can > have > >dates of collection of 19Dec 2004 to 23Jan 2005 and the specimen was from > >some unknown specific date in that rage), > > > >4) continuous data collection over a period of time (some sort of sensor) > > > >5) information collected on one of two possible dates (one or the other > is > >correct, it's not a range) (example: aerial surveys flown on either May > 27 > >or June 04, 1999). > > > >It seems that EarliestCollectingDate / LatestCollectingDate could be used > >for #3, #4, and #5 - but is there a way to differentiate between these as > >actually being very different types of dates? Would those distinctions > just > >be communicated using the VerbatimDateTime? > > > >Of course for all of these examples the fallback would be to use the > >VerbatimDateTime, but it would be ideal to have as much as possible of > the > >date information in field(s) that can be queried and analyzed > > > >Thank you for your thoughts and help. > > > >Lynn > > > > > > > >Lynn Kutner > >NatureServe > >www.natureserve.org > >(303) 541-0360 > >lynn_kutner(a)natureserve.org > > > > > > > >From: Blum, Stan [mailto:sblum@calacademy.org] > >Sent: Thu 2/23/2006 6:41 PM > >To: Lynn Kutner; TDWG(a)LISTSERV.NHM.KU.EDU > >Subject: RE: Standards for date / time values? > > > > > >Hi Lynn, > > > >Context is everything, so I'm going to assume you are talking about > >date/time representations in the observation and monitoring schema(s) > that > >you are developing. The thinking in the collections community has gone > >something along the lines of this: > > > >Concept: the date-time of a collecting event [= recording-event, > >gathering-event] -- i.e., when one or more organisms were collected or > >observed -- expressed in the common Gregorian calendar, in local time. > > > >Requirements: > > > >1 - express the data/time exactly as it was recorded > >2 - retreive records by date/time ranges (date-time greater than X and/or > >less than Y) > >3 - accommodate date-times of varying precision, including explicit > >date-time ranges (specifying a duration) > >4 - support seasonal (time of year) queries > > > >These requirements can be met by the following fields: > > > >VerbatimDateTime > >EarliestDateCollected > >LatestDateCollected > >CollectingDatesInterpreted > >DayOfYear > > > >As John Wieczorek noted, the ISO 8601 format accommodates varying degrees > of > >precision. Using a "verbatim date-time" [text string] is as good as you > can > >do to satisfy the first requirement, short of scanning field notes or > voice > >recordings. The earliest and latest dates support the recording of > explicit > >ranges as well as interpreted ranges, which could be used to make "Summer > of > >1952" retrievable. The CollectingDatesInterpreted field would be a > boolean > >field, set as true when the earliest and latest dates represent > >interpretations rather than an explicit range. The "DayOfYear" field is > a > >compromise that we've offered as a simple way to support queries > involving > >seasonal (annual) cycles; e.g., collected in the northern summer, > regardless > >of year. But it can be argued that day of year is derivable from the > other > >fields (unless year is unknown), and that it doesn't accommodate explicit > >ranges. > > > >A bit more documentation is needed to address the odd cases (What do I do > >when ...?), but these five fields will support a lot of the data exchange > >needed for this concept. These fields are not intended to handle the > dating > >of localities in paleontology, nor are they intended to handle named > periods > >that are used in cultural collections (e.g., Iron Age, Victorian). > > > >ABCD uses a few more fields to handle the concept, > >http://ww3.bgbm.org/abcddocs/AbcdConcepts but some of these (date, time, > >and time zone) are handled by the ISO format, except when the larger > units > >are unkown; .e.g., the year is unknown, but the day is [ June 16 ]; or > the > >date is unknown, but the time is [ 14:00-15:30 ]. > > > >AbcdConcept0860 /DataSets/DataSet/Units/Unit/Gathering/DateTime > >AbcdConcept0861 /DataSets/DataSet/Units/Unit/Gathering/DateTime/DateText > >AbcdConcept0862 /DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeZone > >AbcdConcept0863 > >/DataSets/DataSet/Units/Unit/Gathering/DateTime/ISODateTimeBegin > >AbcdConcept0864 > >/DataSets/DataSet/Units/Unit/Gathering/DateTime/DayNumberBegin > >AbcdConcept0865 > >/DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeOfDayBegin > >AbcdConcept0866 > >/DataSets/DataSet/Units/Unit/Gathering/DateTime/ISODateTimeEnd > >AbcdConcept0867 > >/DataSets/DataSet/Units/Unit/Gathering/DateTime/DayNumberEnd > >AbcdConcept0868 > >/DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeOfDayEnd > >AbcdConcept0869 > >/DataSets/DataSet/Units/Unit/Gathering/DateTime/PeriodExplicit > >The ABCD fields pretty much cover the entire concept space. One could > argue > >whether time-zone is relevant to the description of biological phenomena, > >but we do know that ship trawls do cross time-zones (including the > >date-line), and that daylight savings time could stretch or compress some > >nocturnal collecting events if their durations were calculated too > simply. > > > >To some extent these arguments are still going on, so analyze your > >requirements and your data, then state your position. ;-) > > > >Cheers, > > > >-Stan > >Stanley D. Blum, Ph.D. > >Research Information Manager > >California Academy of Sciences > >875 Howard St. > >San Francisco, CA > >+1 (415) 321-8183 > > > > > >-----Original Message----- > >From: Taxonomic Databases Working Group List > >[mailto:TDWG@LISTSERV.NHM.KU.EDU] On Behalf Of Lynn Kutner > >Sent: Thursday, February 23, 2006 9:12 AM > >To: TDWG(a)LISTSERV.NHM.KU.EDU > >Subject: Standards for date / time values? > > > > > >Hi - > >I'm working with a suite of date attributes that can include a > combination > >of precise dates, imprecise dates, and ranges of dates (and the same > types > >of time values). We'd like to follow existing standards. If this sort of > >date / time standard exists, I'd appreciate leads to the appropriate > >resources. > >Thank you for your help - > >Lynn > > > > > >Lynn Kutner > >Data Management Coordinator > >NatureServe > >Email: lynn_kutner(a)natureserve.org > >Phone: (303) 541-0360 > >www.natureserve.org > > Dra. Patricia Koleff > Directora de An�lisis y Prioridades > CONABIO > Comisi�n Nacional para el Conocimiento y Uso de la Biodiversidad > Av. Liga Periferico - Insurgentes Sur 4903 > Col. Parques del Pedregal > Delegacion Tlalpan > 14010 Mexico, D.F. > MEXICO > Tel. +52 (55) 5528 9105 > Fax +52 (55) 5528 9194 > www.conabio.gob.mx > e-mail: pkoleff(a)xolo.conabio.gob.mx >

1 0

Re: Standards for date / time values?
by Richard Pyle 23 Feb '06

23 Feb '06

RE: CollectingDatesInterpreted When would this field *not* be set to true? In other words, what is the threshold for "interpreted", in terms of populating EarliestCollectingDate and LatestCollectingDate? At one extreme, we have a VerbatimDateTime of "12 January 2005, 13:34:15 local time". Not a lot of interpretation needed there (other than conversion of original documentation format to ASCII/Unicode, and interpretation of the time zone from the locality data). But conversion of "12 January 2005" to EarliestCollectingDate: "2005-01-12 00:00:00" LatestCollectingDate: "2005-01-12 23:59:59" ...requires a bit of interpretation. Maybe the time portion could simply be omitted; but even still, there is technically *some* interpretation when converting "January" to "01". And the spectrum continues on from there up until, well, stuff that would be unambiguously flagged as "interpreted". My point is, there should be some sort of guideline as to when to set CollectingDatesInterpreted true; i.e., what threshold of interpretation triggers that field. If such a guideline already exists, then I apologize for missing it. Personally, I don't think CollectingDatesInterpreted is necessary, really, because I have always regarded my equivalents of EarliestCollectingDate and LatestCollectingDate as interpreted. RE: Lynn's examples below. I wrestled with these issues (and more) a number of years ago. Other examples include "Summer; year unknown" (important for seasonal species), and multiple (non-contiguous) date ranges: "May or August, 2004" -- among others. In addition to the equivalents of VerbatimDateTime, EarliestCollectingDate, LatestCollectingDate; I had a controlled-vocabulary field called "DateRangeQualifier", which had an enumerated list of options tied to interpretation logic (e.g., "continuous range", "discontinuous range", "unknown point", "either/or", etc.; as well as sub-range qualifiers like "Summer"). I modelled it after an analogous field I use for interpreting paired or multiple georef coordinates (e.g., "transect" vs. "bounding box"; "polygon" vs. "linear"). This DateRangeQualifier approach ended up being too cumbersome to implement -- only because I was too lazy, and the number of cases that benefited from it (i.e., allowed computer logic to be applied, independent of a human reading of the VerbatimDateTime value) was low. The three fields VerbatimDateTime/EarliestCollectingDate/LatestCollectingDate will give you 90% of the value 90% of the time (at least for the specimen/observation data that I deal with). I wonder if DWC really needs more precision capability than that.... Rich Richard L. Pyle, PhD Database Coordinator for Natural Sciences Department of Natural Sciences, Bishop Museum 1525 Bernice St., Honolulu, HI 96817 Ph: (808)848-4115, Fax: (808)847-8252 email: deepreef(a)bishopmuseum.org http://hbs.bishopmuseum.org/staff/pylerichard.html -----Original Message----- From: Taxonomic Databases Working Group List [mailto:TDWG@LISTSERV.NHM.KU.EDU]On Behalf Of Lynn Kutner Sent: Thursday, February 23, 2006 5:26 PM To: TDWG(a)LISTSERV.NHM.KU.EDU Subject: Re: Standards for date / time values? Hi Stan and all - Yes, I am thinking about dates in the context of the observation / monitoring schema. Here are some of the types of dates that I've seen in just the data from the NatureServe member programs based on either lack of available information or interesting data collection situations. 1) partial dates (only have the year or year-month) - definitely accommodated by the ISO 8601 format 2) vague dates - for these I suppose you'd use ISO 8601 in EarliestCollectingDate / LatestCollectingDate, VerbatimDateTime, and the CollectingDatesInterpreted flag ? PRE-1975 post-1992 summer 2001 Mid-1990s late 1950's Circa 1941 Early 1990s 3) a range of dates indicating that collection occurred on a specific date sometime between those two limits and you're not sure when (example: Observations / collections from pitfall traps may have date ranges associated with them rather than a single date. For instance, you can have dates of collection of 19Dec 2004 to 23Jan 2005 and the specimen was from some unknown specific date in that rage), 4) continuous data collection over a period of time (some sort of sensor) 5) information collected on one of two possible dates (one or the other is correct, it's not a range) (example: aerial surveys flown on either May 27 or June 04, 1999). It seems that EarliestCollectingDate / LatestCollectingDate could be used for #3, #4, and #5 - but is there a way to differentiate between these as actually being very different types of dates? Would those distinctions just be communicated using the VerbatimDateTime? Of course for all of these examples the fallback would be to use the VerbatimDateTime, but it would be ideal to have as much as possible of the date information in field(s) that can be queried and analyzed Thank you for your thoughts and help. Lynn Lynn Kutner NatureServe www.natureserve.org (303) 541-0360 lynn_kutner(a)natureserve.org From: Blum, Stan [mailto:sblum@calacademy.org] Sent: Thu 2/23/2006 6:41 PM To: Lynn Kutner; TDWG(a)LISTSERV.NHM.KU.EDU Subject: RE: Standards for date / time values? Hi Lynn, Context is everything, so I'm going to assume you are talking about date/time representations in the observation and monitoring schema(s) that you are developing. The thinking in the collections community has gone something along the lines of this: Concept: the date-time of a collecting event [= recording-event, gathering-event] -- i.e., when one or more organisms were collected or observed -- expressed in the common Gregorian calendar, in local time. Requirements: 1 - express the data/time exactly as it was recorded 2 - retreive records by date/time ranges (date-time greater than X and/or less than Y) 3 - accommodate date-times of varying precision, including explicit date-time ranges (specifying a duration) 4 - support seasonal (time of year) queries These requirements can be met by the following fields: VerbatimDateTime EarliestDateCollected LatestDateCollected CollectingDatesInterpreted DayOfYear As John Wieczorek noted, the ISO 8601 format accommodates varying degrees of precision. Using a "verbatim date-time" [text string] is as good as you can do to satisfy the first requirement, short of scanning field notes or voice recordings. The earliest and latest dates support the recording of explicit ranges as well as interpreted ranges, which could be used to make "Summer of 1952" retrievable. The CollectingDatesInterpreted field would be a boolean field, set as true when the earliest and latest dates represent interpretations rather than an explicit range. The "DayOfYear" field is a compromise that we've offered as a simple way to support queries involving seasonal (annual) cycles; e.g., collected in the northern summer, regardless of year. But it can be argued that day of year is derivable from the other fields (unless year is unknown), and that it doesn't accommodate explicit ranges. A bit more documentation is needed to address the odd cases (What do I do when ...?), but these five fields will support a lot of the data exchange needed for this concept. These fields are not intended to handle the dating of localities in paleontology, nor are they intended to handle named periods that are used in cultural collections (e.g., Iron Age, Victorian). ABCD uses a few more fields to handle the concept, http://ww3.bgbm.org/abcddocs/AbcdConcepts but some of these (date, time, and time zone) are handled by the ISO format, except when the larger units are unkown; .e.g., the year is unknown, but the day is [ June 16 ]; or the date is unknown, but the time is [ 14:00-15:30 ]. AbcdConcept0860 /DataSets/DataSet/Units/Unit/Gathering/DateTime AbcdConcept0861 /DataSets/DataSet/Units/Unit/Gathering/DateTime/DateText AbcdConcept0862 /DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeZone AbcdConcept0863 /DataSets/DataSet/Units/Unit/Gathering/DateTime/ISODateTimeBegin AbcdConcept0864 /DataSets/DataSet/Units/Unit/Gathering/DateTime/DayNumberBegin AbcdConcept0865 /DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeOfDayBegin AbcdConcept0866 /DataSets/DataSet/Units/Unit/Gathering/DateTime/ISODateTimeEnd AbcdConcept0867 /DataSets/DataSet/Units/Unit/Gathering/DateTime/DayNumberEnd AbcdConcept0868 /DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeOfDayEnd AbcdConcept0869 /DataSets/DataSet/Units/Unit/Gathering/DateTime/PeriodExplicit The ABCD fields pretty much cover the entire concept space. One could argue whether time-zone is relevant to the description of biological phenomena, but we do know that ship trawls do cross time-zones (including the date-line), and that daylight savings time could stretch or compress some nocturnal collecting events if their durations were calculated too simply. To some extent these arguments are still going on, so analyze your requirements and your data, then state your position. ;-) Cheers, -Stan Stanley D. Blum, Ph.D. Research Information Manager California Academy of Sciences 875 Howard St. San Francisco, CA +1 (415) 321-8183 -----Original Message----- From: Taxonomic Databases Working Group List [mailto:TDWG@LISTSERV.NHM.KU.EDU] On Behalf Of Lynn Kutner Sent: Thursday, February 23, 2006 9:12 AM To: TDWG(a)LISTSERV.NHM.KU.EDU Subject: Standards for date / time values? Hi - I'm working with a suite of date attributes that can include a combination of precise dates, imprecise dates, and ranges of dates (and the same types of time values). We'd like to follow existing standards. If this sort of date / time standard exists, I'd appreciate leads to the appropriate resources. Thank you for your help - Lynn Lynn Kutner Data Management Coordinator NatureServe Email: lynn_kutner(a)natureserve.org Phone: (303) 541-0360 www.natureserve.org

1 0

Re: Standards for date / time values?
by Lynn Kutner 23 Feb '06

23 Feb '06

Hi Stan and all - Yes, I am thinking about dates in the context of the observation / monitoring schema. Here are some of the types of dates that I've seen in just the data from the NatureServe member programs based on either lack of available information or interesting data collection situations. 1) partial dates (only have the year or year-month) - definitely accommodated by the ISO 8601 format 2) vague dates - for these I suppose you'd use ISO 8601 in EarliestCollectingDate / LatestCollectingDate, VerbatimDateTime, and the CollectingDatesInterpreted flag ? PRE-1975 post-1992 summer 2001 Mid-1990s late 1950's Circa 1941 Early 1990s 3) a range of dates indicating that collection occurred on a specific date sometime between those two limits and you're not sure when (example: Observations / collections from pitfall traps may have date ranges associated with them rather than a single date. For instance, you can have dates of collection of 19Dec 2004 to 23Jan 2005 and the specimen was from some unknown specific date in that rage), 4) continuous data collection over a period of time (some sort of sensor) 5) information collected on one of two possible dates (one or the other is correct, it's not a range) (example: aerial surveys flown on either May 27 or June 04, 1999). It seems that EarliestCollectingDate / LatestCollectingDate could be used for #3, #4, and #5 - but is there a way to differentiate between these as actually being very different types of dates? Would those distinctions just be communicated using the VerbatimDateTime? Of course for all of these examples the fallback would be to use the VerbatimDateTime, but it would be ideal to have as much as possible of the date information in field(s) that can be queried and analyzed Thank you for your thoughts and help. Lynn Lynn Kutner NatureServe www.natureserve.org (303) 541-0360 lynn_kutner(a)natureserve.org ________________________________ From: Blum, Stan [mailto:sblum@calacademy.org] Sent: Thu 2/23/2006 6:41 PM To: Lynn Kutner; TDWG(a)LISTSERV.NHM.KU.EDU Subject: RE: Standards for date / time values? Hi Lynn, Context is everything, so I'm going to assume you are talking about date/time representations in the observation and monitoring schema(s) that you are developing. The thinking in the collections community has gone something along the lines of this: Concept: the date-time of a collecting event [= recording-event, gathering-event] -- i.e., when one or more organisms were collected or observed -- expressed in the common Gregorian calendar, in local time. Requirements: 1 - express the data/time exactly as it was recorded 2 - retreive records by date/time ranges (date-time greater than X and/or less than Y) 3 - accommodate date-times of varying precision, including explicit date-time ranges (specifying a duration) 4 - support seasonal (time of year) queries These requirements can be met by the following fields: VerbatimDateTime EarliestDateCollected LatestDateCollected CollectingDatesInterpreted DayOfYear As John Wieczorek noted, the ISO 8601 format accommodates varying degrees of precision. Using a "verbatim date-time" [text string] is as good as you can do to satisfy the first requirement, short of scanning field notes or voice recordings. The earliest and latest dates support the recording of explicit ranges as well as interpreted ranges, which could be used to make "Summer of 1952" retrievable. The CollectingDatesInterpreted field would be a boolean field, set as true when the earliest and latest dates represent interpretations rather than an explicit range. The "DayOfYear" field is a compromise that we've offered as a simple way to support queries involving seasonal (annual) cycles; e.g., collected in the northern summer, regardless of year. But it can be argued that day of year is derivable from the other fields (unless year is unknown), and that it doesn't accommodate explicit ranges. A bit more documentation is needed to address the odd cases (What do I do when ...?), but these five fields will support a lot of the data exchange needed for this concept. These fields are not intended to handle the dating of localities in paleontology, nor are they intended to handle named periods that are used in cultural collections (e.g., Iron Age, Victorian). ABCD uses a few more fields to handle the concept, http://ww3.bgbm.org/abcddocs/AbcdConcepts <http://ww3.bgbm.org/abcddocs/AbcdConcepts> but some of these (date, time, and time zone) are handled by the ISO format, except when the larger units are unkown; .e.g., the year is unknown, but the day is [ June 16 ]; or the date is unknown, but the time is [ 14:00-15:30 ]. * AbcdConcept0860 <http://ww3.bgbm.org/abcddocs/AbcdConcept0860> /DataSets/DataSet/Units/Unit/Gathering/DateTime * AbcdConcept0861 <http://ww3.bgbm.org/abcddocs/AbcdConcept0861> /DataSets/DataSet/Units/Unit/Gathering/DateTime/DateText * AbcdConcept0862 <http://ww3.bgbm.org/abcddocs/AbcdConcept0862> /DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeZone * AbcdConcept0863 <http://ww3.bgbm.org/abcddocs/AbcdConcept0863> /DataSets/DataSet/Units/Unit/Gathering/DateTime/ISODateTimeBegin * AbcdConcept0864 <http://ww3.bgbm.org/abcddocs/AbcdConcept0864> /DataSets/DataSet/Units/Unit/Gathering/DateTime/DayNumberBegin * AbcdConcept0865 <http://ww3.bgbm.org/abcddocs/AbcdConcept0865> /DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeOfDayBegin * AbcdConcept0866 <http://ww3.bgbm.org/abcddocs/AbcdConcept0866> /DataSets/DataSet/Units/Unit/Gathering/DateTime/ISODateTimeEnd * AbcdConcept0867 <http://ww3.bgbm.org/abcddocs/AbcdConcept0867> /DataSets/DataSet/Units/Unit/Gathering/DateTime/DayNumberEnd * AbcdConcept0868 <http://ww3.bgbm.org/abcddocs/AbcdConcept0868> /DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeOfDayEnd * AbcdConcept0869 <http://ww3.bgbm.org/abcddocs/AbcdConcept0869> /DataSets/DataSet/Units/Unit/Gathering/DateTime/PeriodExplicit The ABCD fields pretty much cover the entire concept space. One could argue whether time-zone is relevant to the description of biological phenomena, but we do know that ship trawls do cross time-zones (including the date-line), and that daylight savings time could stretch or compress some nocturnal collecting events if their durations were calculated too simply. To some extent these arguments are still going on, so analyze your requirements and your data, then state your position. ;-) Cheers, -Stan Stanley D. Blum, Ph.D. Research Information Manager California Academy of Sciences 875 Howard St. San Francisco, CA +1 (415) 321-8183 -----Original Message----- From: Taxonomic Databases Working Group List [mailto:TDWG@LISTSERV.NHM.KU.EDU] On Behalf Of Lynn Kutner Sent: Thursday, February 23, 2006 9:12 AM To: TDWG(a)LISTSERV.NHM.KU.EDU Subject: Standards for date / time values? Hi - I'm working with a suite of date attributes that can include a combination of precise dates, imprecise dates, and ranges of dates (and the same types of time values). We'd like to follow existing standards. If this sort of date / time standard exists, I'd appreciate leads to the appropriate resources. Thank you for your help - Lynn Lynn Kutner Data Management Coordinator NatureServe Email: lynn_kutner(a)natureserve.org Phone: (303) 541-0360 www.natureserve.org <file://www.natureserve.org>

1 0

Re: Standards for date / time values?
by Gail Kampmeier 23 Feb '06

23 Feb '06

Hi Lynn & John, Personally, I don't think that combined date fields are adequate, particularly for retrospective data capture. We have far too many insect labels with partial dates (tiny labels, tiny print, no room), which really mess up (probably also platform dependent screwups) when you try to enter the data as a date without one of the pieces. We've found all permutations--year only, month and year, month and day, well, maybe not day and year...but that's enough of a variety that we have all the fields individually broken out to accomodate this and then account for missing data in a calculated "pseudo" date by making assumptions depending on which piece may be missing. It also gives you greater flexibility and assuredness that you have the day and the month able to be put in the order desired or show the date in different formats such as 6.ix.1956, where the day and the month could easily be confused. Cheers! Gail >Hi Lynn, > >XML Schema, in its dateTime type, supports a subset of the ISO 8601 >datetime standard. The proposal for DwC2 currently recommends the >concepts EarliestDateCollected and LatestDateCollected, both as >dateTime types, to capture bounded or unbounded date/time >combinations of arbitrary specificity. These should be sufficient >except where lists of dates or other multiple date ranges are >required for a single record - unless I'm missing something, in >which case I'd like to know. > >Cheers, > >John > > >On 2/23/06, Lynn Kutner ><<mailto:Lynn_Kutner@natureserve.org>Lynn_Kutner(a)natureserve.org> >wrote: > >Hi - > >I'm working with a suite of date attributes that can include a >combination of precise dates, imprecise dates, and ranges of dates >(and the same types of time values). We'd like to follow existing >standards. If this sort of date / time standard exists, I'd >appreciate leads to the appropriate resources. > >Thank you for your help - >Lynn > > >Lynn Kutner >Data Management Coordinator >NatureServe >Email: <mailto:lynn_kutner@natureserve.org>lynn_kutner(a)natureserve.org >Phone: (303) 541-0360 >www.natureserve.org -- ====================== Gail E. Kampmeier Sr. Research Entomologist Illinois Natural History Survey Box 5 NSRC, 1101 W. Peabody Dr. Urbana, IL 61801 http://www.inhs.uiuc.edu/~gkamp ======================

1 0

Re: Standards for date / time values?
by Blum, Stan 23 Feb '06

23 Feb '06

Hi Lynn, Context is everything, so I'm going to assume you are talking about date/time representations in the observation and monitoring schema(s) that you are developing. The thinking in the collections community has gone something along the lines of this: Concept: the date-time of a collecting event [= recording-event, gathering-event] -- i.e., when one or more organisms were collected or observed -- expressed in the common Gregorian calendar, in local time. Requirements: 1 - express the data/time exactly as it was recorded 2 - retreive records by date/time ranges (date-time greater than X and/or less than Y) 3 - accommodate date-times of varying precision, including explicit date-time ranges (specifying a duration) 4 - support seasonal (time of year) queries These requirements can be met by the following fields: VerbatimDateTime EarliestDateCollected LatestDateCollected CollectingDatesInterpreted DayOfYear As John Wieczorek noted, the ISO 8601 format accommodates varying degrees of precision. Using a "verbatim date-time" [text string] is as good as you can do to satisfy the first requirement, short of scanning field notes or voice recordings. The earliest and latest dates support the recording of explicit ranges as well as interpreted ranges, which could be used to make "Summer of 1952" retrievable. The CollectingDatesInterpreted field would be a boolean field, set as true when the earliest and latest dates represent interpretations rather than an explicit range. The "DayOfYear" field is a compromise that we've offered as a simple way to support queries involving seasonal (annual) cycles; e.g., collected in the northern summer, regardless of year. But it can be argued that day of year is derivable from the other fields (unless year is unknown), and that it doesn't accommodate explicit ranges. A bit more documentation is needed to address the odd cases (What do I do when ...?), but these five fields will support a lot of the data exchange needed for this concept. These fields are not intended to handle the dating of localities in paleontology, nor are they intended to handle named periods that are used in cultural collections (e.g., Iron Age, Victorian). ABCD uses a few more fields to handle the concept, http://ww3.bgbm.org/abcddocs/AbcdConcepts <http://ww3.bgbm.org/abcddocs/AbcdConcepts> but some of these (date, time, and time zone) are handled by the ISO format, except when the larger units are unkown; .e.g., the year is unknown, but the day is [ June 16 ]; or the date is unknown, but the time is [ 14:00-15:30 ]. * AbcdConcept0860 <http://ww3.bgbm.org/abcddocs/AbcdConcept0860> /DataSets/DataSet/Units/Unit/Gathering/DateTime * AbcdConcept0861 <http://ww3.bgbm.org/abcddocs/AbcdConcept0861> /DataSets/DataSet/Units/Unit/Gathering/DateTime/DateText * AbcdConcept0862 <http://ww3.bgbm.org/abcddocs/AbcdConcept0862> /DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeZone * AbcdConcept0863 <http://ww3.bgbm.org/abcddocs/AbcdConcept0863> /DataSets/DataSet/Units/Unit/Gathering/DateTime/ISODateTimeBegin * AbcdConcept0864 <http://ww3.bgbm.org/abcddocs/AbcdConcept0864> /DataSets/DataSet/Units/Unit/Gathering/DateTime/DayNumberBegin * AbcdConcept0865 <http://ww3.bgbm.org/abcddocs/AbcdConcept0865> /DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeOfDayBegin * AbcdConcept0866 <http://ww3.bgbm.org/abcddocs/AbcdConcept0866> /DataSets/DataSet/Units/Unit/Gathering/DateTime/ISODateTimeEnd * AbcdConcept0867 <http://ww3.bgbm.org/abcddocs/AbcdConcept0867> /DataSets/DataSet/Units/Unit/Gathering/DateTime/DayNumberEnd * AbcdConcept0868 <http://ww3.bgbm.org/abcddocs/AbcdConcept0868> /DataSets/DataSet/Units/Unit/Gathering/DateTime/TimeOfDayEnd * AbcdConcept0869 <http://ww3.bgbm.org/abcddocs/AbcdConcept0869> /DataSets/DataSet/Units/Unit/Gathering/DateTime/PeriodExplicit The ABCD fields pretty much cover the entire concept space. One could argue whether time-zone is relevant to the description of biological phenomena, but we do know that ship trawls do cross time-zones (including the date-line), and that daylight savings time could stretch or compress some nocturnal collecting events if their durations were calculated too simply. To some extent these arguments are still going on, so analyze your requirements and your data, then state your position. ;-) Cheers, -Stan Stanley D. Blum, Ph.D. Research Information Manager California Academy of Sciences 875 Howard St. San Francisco, CA +1 (415) 321-8183 -----Original Message----- From: Taxonomic Databases Working Group List [mailto:TDWG@LISTSERV.NHM.KU.EDU] On Behalf Of Lynn Kutner Sent: Thursday, February 23, 2006 9:12 AM To: TDWG(a)LISTSERV.NHM.KU.EDU Subject: Standards for date / time values? Hi - I'm working with a suite of date attributes that can include a combination of precise dates, imprecise dates, and ranges of dates (and the same types of time values). We'd like to follow existing standards. If this sort of date / time standard exists, I'd appreciate leads to the appropriate resources. Thank you for your help - Lynn Lynn Kutner Data Management Coordinator NatureServe Email: lynn_kutner(a)natureserve.org Phone: (303) 541-0360 www.natureserve.org <file://www.natureserve.org>

1 0

Re: Standards for date / time values?
by John R. WIECZOREK 23 Feb '06

23 Feb '06

Hi Lynn, XML Schema, in its dateTime type, supports a subset of the ISO 8601 datetime standard. The proposal for DwC2 currently recommends the concepts EarliestDateCollected and LatestDateCollected, both as dateTime types, to capture bounded or unbounded date/time combinations of arbitrary specificity. These should be sufficient except where lists of dates or other multiple date ranges are required for a single record - unless I'm missing something, in which case I'd like to know. Cheers, John On 2/23/06, Lynn Kutner <Lynn_Kutner(a)natureserve.org> wrote: > > Hi - > > I'm working with a suite of date attributes that can include a combination > of precise dates, imprecise dates, and ranges of dates (and the same types > of time values). We'd like to follow existing standards. If this sort of > date / time standard exists, I'd appreciate leads to the appropriate > resources. > > Thank you for your help - > Lynn > > Lynn Kutner > *Data Management Coordinator* > *NatureServe* > Email: lynn_kutner(a)natureserve.org > Phone: (303) 541-0360 > ***www.natureserve.org* >

1 0

Standards for date / time values?
by Lynn Kutner 23 Feb '06

23 Feb '06

Hi - I'm working with a suite of date attributes that can include a combination of precise dates, imprecise dates, and ranges of dates (and the same types of time values). We'd like to follow existing standards. If this sort of date / time standard exists, I'd appreciate leads to the appropriate resources. Thank you for your help - Lynn Lynn Kutner Data Management Coordinator NatureServe Email: lynn_kutner(a)natureserve.org Phone: (303) 541-0360 www.natureserve.org

1 0

TDWG/GBIF GUID-1 Workshop Report
by Ricardo Scachetti Pereira 13 Feb '06

13 Feb '06

TDWG/GBIF GUID-1 Workshop Report ================================ The Taxonomic Databases Working Group (TDWG) and the Global Biodiversity Information Facility (GBIF) completed their first Workshop on Globally Unique Identifiers for Biodiversity Informatics (GUID-1) at the National Center for Evolutionary Synthesis (NESCent), Durham, NC, USA on Feb 1-3, 2006. Motivation ========== A GUID framework is foundational in facilitating systems interoperability in biodiversity informatics. It meets the need for a universally adopted system for assigning and recognizing identifiers in the domain. A GUID system will help to manage and cross-link the many different types of entities that are manipulated analytically in biodiversity informatics and will improve interoperability with other related life sciences domains, such as bioinformatics and ecology. The Group ========= The workshop delegates consisted of a representative cross-section of domain experts from around the world (see http://wiki.gbif.org/guidwiki/wikka.php?wakka=GUID1Participants). Goals ===== The goals of the workshop were to: * Discuss the requirements for globally unique identifiers for biodiversity informatics * Select an optimal GUID technology (LSID, DOI, Handles or other) * Begin to identify key parameters for implementing an effective system * Investigate the use of a RDF-based metadata architecture for GUIDs * Form working groups to address key identified issues before the GUID-2 workshop Outcomes ======== * Life Science Identifiers (LSID) seem the most appropriate GUID strategy in biodiversity informatics. * The use of LSIDs does not preclude the use of other technologies where appropriate. * LSID authorities must use the Domain Name Service (DNS) to support identifier resolution. (The LSID specification allows for other resolution mechanisms, but DNS is currently the only mechanism in use.) * Although it is not possible to prevent multiple data providers from issuing alternate identifiers resolving to the same data record, the community should develop processes and tools to coordinate issuing of single identifiers for some classes of data (e.g. taxon names). * Metadata should be provided as RDF serialized as XML and should exploit existing vocabularies such as Dublin Core wherever these are in wide use. * The LSID getData method should be used only where it is possible and appropriate to return an unchanging series of bytes. In other cases only the LSID getMetadata method should be used. (This reflects the use of the terms “data” and “metadata” in connection with LSIDs.) Justifications ============== The main criteria leading to the selection of LSID technology were: * The cost-model of DOI. That technology is predicated on the idea that a revenue stream can be constructed for the identified objects, typically sufficient to defray the cost. That this is not the case for most, if not all, of the objects that are likely to be identified in our systems. * The more dynamic nature of LSIDs, which does not require prior registration of every individual identifier before use. * The open nature of the LSID protocol and software stack, and the ease of implementing LSIDs on different platforms. See http://wiki.gbif.org/guidwiki/wikka.php?wakka=GUID1Report for a more detailed comparison of GUID technologies. Work Plan ========= There are still many issues to address before our community can fully implement an identifier system based on LSIDs. The workshop addressed a number of specific issues and developed working groups to address the following issues: * Developing white papers to address best practices and key infrastructure questions. * Prototyping activities. The Infrastructure Working Group ================================ This group was formed to address the key issues regarding the deployment of LSID as the GUID technology for biodiversity informatics. The mandate of this working group is to identify required or desirable policies and infrastructure components to ensure robust, long-term operation of shared GUIDs. The following activities were identified: 1. Specify minimal standards (including tools and services) for GUID issuance. 2. Investigate long-term archival of LSIDs and associated data and metadata. 3. Investigate establishing (optional) central registration authority. 4. Investigate establishing repository for data and (orphan) datasets with GUIDs 5. Investigate the feasibility, existing actors and requirements for a ”Publication Bank” (a resource to act as a central registry of taxonomic literature and its digital representations, including assigning GUIDs to each publication. 6. Clarify the distinction between GUIDs assigned to data objects and to conceptual entities. 7. Investigate 3rd-party annotation and link-out mechanisms. 8. Develop materials to communicate with wider community. 9. Develop best practices for assigning resolver namespaces for LSIDs. 10. Perform review of LSID specification to identify possible enhancements. 11. Perform gap analysis of LSID software.. The outcomes of this group will be a series of white papers addressing the key infrastructure issues. Those will be reviewed during the second GUID meeting later this year. The Prototyping Working Group ============================= Our community must experiment with LSID technology and Ontology Engineering if we are to implement a production quality LSID system, The working group will develop prototypes of test cases to test aspects of a GUID infrastructure. The group will develop test LSID resolvers using data objects provided by each domain, such as names, specimens, and concepts. This activity will also help involve (and train) the community in developing appropriate RDF ontologies, leading to concrete recommendations and implementations. The potential prototypes to be developed and respective Conveners are: 1. LSID resolver for taxon names – developed by nomenclators using IPNI database and an RDF version of TCS-Names – Group responsible: Roger Hyam, Sally Hinchcliffe, Paul Kirk 2. LSID resolver for specimens using DarwinCore (also ABCD?) - Steve Perry. 3. LSID resolver for taxon concepts by SEEK using TCS. 4. LSID resolver for observations by SEEK using EML. 5. LSID resolver for character data by Damian Barnier, Kevin Thiele 6. LSID resolver for images: Greg Riccardi (MorphBank), Bob Morris The taxon names resolver has the highest priority. Prototypes will address one or more of the following (but may not be full implementations of an LSID resolution service): * Hardware and software (including LSID stack) * RDFS/OWL vocabulary for domain * Data mapping between local data store scheme and shared ontology Other important tasks identified by this group are: * Development of ontologies to represent metadata for the various domains. Coordinated by TDWG TAG with help of experienced ontology engineers. * To set up a real live LSID server to perform scalability testing. * A project to demonstrate the potential from LSID-based integration of data for a particular group (Ants) – LSIDs, taxonomic lit, specimen, images, names, sequences from Genbank – Rod Page * To use SEEK Taxon resolution server (alpha) for testing. This group will have 3 months to work on the specified tasks before preparing for the second GUID workshop. Next Workshop: GUID-2 ===================== The TDWG Infrastructure Project is planning a second GUID workshop in late May or early June, 2006. At the time of writing a venue has not been decided. The second workshop should cover the following: * Review of the material produced between the workshops by both working groups (prototypes and white papers). * Summary of the lessons learned in the process. * Identify open issues and devise specific work plans to address them. * Draft concrete recommendations on GUIDs for production systems – in general and for each specific domain (names, specimens, concepts, images, etc). Information about GUID-2 will be distributed as soon as possible. Other Resources =============== * See http://wiki.gbif.org/guidwiki for more on the TDWG GUID effort. * See http://wiki.gbif.org/guidwiki/wikka.php?wakka=GUID1Minutes for the workshop minutes and presentations. * See http://wiki.gbif.org/guidwiki/wikka.php?wakka=GUID1Report for a full report, or download it in pdf format from the following link: http://wiki.gbif.org/guidwiki/images/GUID-1Report.pdf -------------------- Ricardo Pereira Software Engineer Taxonomic Databases Working Group (TDWG) TDWG Infrastructure Project (TIP)

1 0