Subject: Re: [Tzdist-bis] annotated idnits for draft-murchison-tzdist-tzif-09

From: Ken Murchison <[email-masked]>
Date: Fri, 3 Aug 2018 15:14:47 -0400

Getting back into the truncation issue after a llong break  What data in 
TZif can NOT be represented in iCalendar?

On 07/02/2018 04:55 PM, Paul Eggert wrote:
> Paul Eggert wrote:
>>
>> neither TZif nor VTIMEZONE dominates the other. They both can have 
>> info that the other format can't represent.
>
> This point is worth putting directly into the spec, as part of the 
> introduction. Proposed patch attached.
>
> tzif-09-5.diff
>
>
> --- draft-murchison-tzdist-tzif-09-4.xml	2018-07-01 11:40:11.878743855 -0700
> +++ draft-murchison-tzdist-tzif-09-5.xml	2018-07-02 13:29:05.121756694 -0700
> @@ -112,7 +112,10 @@
> 	      time.  There is a wide variety of interoperable <xref
> 	      target="tz-link">software</xref> capable of generating and
> 	      reading files in this format.
> -	      </t>
> +	      TZif conveys information similar to that conveyed by the
> +	      VTIMEZONE iCalendar component, although neither format dominates
> +	      the other (that is, either format can contain information that
> +	      cannot be represented in the other format).</t>
> 	
> 	      <t>This specification does not define the source of the time zone
> 	      data or leap second information.

-- 
Ken Murchison
Cyrus Development Team
FastMail US LLC

---

From: Paul Eggert <[email-masked]>
Date: Fri, 3 Aug 2018 13:56:50 -0700

On 08/03/2018 12:14 PM, Ken Murchison wrote:
> What data in TZif can NOT be represented in iCalendar?

I don't have a complete list. However, the standard/wall flags don't
have an iCalendar counterpart. Also, iCalendar cannot handle dates
before the year 0 or after 9999 (admittedly not a real problem in
practice yet).  And can iCalendar represent everything in an extended
POSIX TZ string can? I expect not, though I haven't checked. Maybe leap
seconds too?

---

From: Ken Murchison <[email-masked]>
Date: Fri, 3 Aug 2018 18:20:36 -0400

On 08/03/2018 04:56 PM, Paul Eggert wrote:
> have an iCalendar counterpart.

Yes and no.  Since (unlike TZif) iCalendar is extensible, my forked copy 
if vzic (now part of libical) adds a parameter to DTSTART and RDATE 
properties specifying whether the observance is UTC or standard time 
(absence of the property is wall time).

>   Also, iCalendar cannot handle dates
> practice yet).
True.

> And can iCalendar represent everything in an extended
> POSIX TZ string can? I expect not, though I haven't checked.

I haven't come across an extended TZ string that couldn't be represented 
by an iCalendar RRULE.  If you have one in mind, I'd be curious to check.

> Maybe leap

Its definitely doesn't represent leap seconds embedded in a given 
VTIMEZONE component, but we could generate a separate VTIMEZONE 
representing TAI - I think one was proposed on the list but TZDIST chose 
to use a JSON resource to represent leap seconds..  I rather like the 
idea of keeping leap seconds outside of the time zone data, because its 
universal across all time zones, and just adds bloat to a full TZif distro.

-- 
Ken Murchison
Cyrus Development Team
FastMail US LLC

---

From: Paul Eggert <[email-masked]>
Date: Fri, 3 Aug 2018 16:51:02 -0700

Ken Murchison wrote:

>> However, the standard/wall flags don't
> Yes and no.  Since (unlike TZif) iCalendar is extensible

That's a bit harsh. I mean, TZif is extensible too; just go and extend it and 
update the version byte. Realistically, any extension to represent this part of 
TZif is nonstandard and not likely to be observed by other implementations, so 
as a practical matter this part of TZif isn't really covered by iCalendar.

> I haven't come across an extended TZ string that couldn't be represented by an 
> iCalendar RRULE.  If you have one in mind, I'd be curious to check.

I was thinking of the extensions to POSIX involving negative time-of-day, that 
sort of thing. I don't see where iCalendar does that, though I'm no iCalendar 
expert.

> I rather like the idea of keeping leap seconds outside 
> of the time zone data, because its universal across all time zones, and just 
> adds bloat to a full TZif distro.

Yes, in hindsight that may well have been a better way to go. But we're kind of 
stuck with it now. And an advantage is that it's more efficient in the typical 
case. Anyway, it's something TZif has that iCalendar doesn't.

---

From: Ken Murchison <[email-masked]>
Date: Fri, 3 Aug 2018 20:59:19 -0400

On 08/03/2018 07:51 PM, Paul Eggert wrote:
> That's a bit harsh. I mean, TZif is extensible too; just go and extend 
> it and update the version byte. Realistically, any extension to 
> represent this part of TZif is nonstandard and not likely to be 
> observed by other implementations, so as a practical matter this part 
> of TZif isn't really covered by iCalendar.

I didn't intend to bash TZif.  I was merely pointing out (albeit poorly) 
that additional metadata can be added to iCalendar without bumping the 
version.  Existing implementations are required to ignore, but pass 
along, unknown components/properties/parameters.

>
>> I haven't come across an extended TZ string that couldn't be 
>> represented by an iCalendar RRULE.  If you have one in mind, I'd be 
>> curious to check.
>
> I was thinking of the extensions to POSIX involving negative 
> time-of-day, that sort of thing. I don't see where iCalendar does 
> that, though I'm no iCalendar expert.

Right, and I'm curious if negative time of day is the only way to 
represent such rules.  iCalendar RRULEs can represent a whole bunch of 
crazy shit.  Do you know OTH of a time zone which uses such a TZ 
string?  I'd like to look up the corresponding iCalendar that my version 
of vzic outputs.

>
>> I rather like the idea of keeping leap seconds outside of the time 
>> zone data, because its universal across all time zones, and just adds 
>> bloat to a full TZif distro.
>
> Yes, in hindsight that may well have been a better way to go. But 
> we're kind of stuck with it now. And an advantage is that it's more 
> efficient in the typical case. Anyway, it's something TZif has that 
> iCalendar doesn't.

It *may* be possible to include a TAI VTIMEZONE in the same iCalendar 
file as a regular VTIMEZONE (e.g. America/New_York).  But, I doubt any 
app using iCalendar would care or use such data.

-- 
Ken Murchison
Cyrus Development Team
FastMail US LLC

---

From: Paul Eggert <[email-masked]>
Date: Fri, 3 Aug 2018 18:15:28 -0700

Ken Murchison wrote:

> I'm curious if negative time of day is the only way to represent such 
> rules.  iCalendar RRULEs can represent a whole bunch of crazy shit.  Do you know 
> OTH of a time zone which uses such a TZ string?  I'd like to look up the 
> corresponding iCalendar that my version of vzic outputs.

The only one in current use is America/Godthab, which uses 
TZ='<-03>3<-02>,M3.5.0/-2,M10.5.0/-1'. Formerly I dimly recall there being 
others with time-of-day greater than 24:00, but unfortunately I don't remember 
the details. It was probably near the International Date Line. Plus, we used to 
have a permanent-DST zone, but I expect iCalendar can handle that (even though 
POSIX cannot).

While we're on the topic, I assume that iCalendar can handle Irish time even 
though it uses negative DST. This is TZ='IST-1GMT0,M10.5.0,M3.5.0/1'. It's not 
an extension to POSIX; it is pure POSIX (though some software has trouble with it).

---

From: Ken Murchison <[email-masked]>
Date: Fri, 3 Aug 2018 22:47:33 -0400

On 08/03/2018 09:15 PM, Paul Eggert wrote:
>> rules.  iCalendar RRULEs can represent a whole bunch of crazy shit.  
>> Do you know OTH of a time zone which uses such a TZ string?  I'd like 
>> to look up the corresponding iCalendar that my version of vzic outputs.
>
> TZ='<-03>3<-02>,M3.5.0/-2,M10.5.0/-1'. 

My vzic represents this as:

BEGIN:DAYLIGHT
TZNAME:-02
TZOFFSETFROM:-0300
TZOFFSETTO:-0200
DTSTART;X-OBSERVED-AT=Z:19810328T220000
RRULE:FREQ=YEARLY;BYMONTH=3;BYMONTHDAY=24,25,26,27,28,29,30;BYDAY=SA
END:DAYLIGHT

BEGIN:STANDARD
TZNAME:-03
TZOFFSETFROM:-0200
TZOFFSETTO:-0300
DTSTART;X-OBSERVED-AT=Z:19961026T230000
RRULE:FREQ=YEARLY;BYMONTH=10;BYMONTHDAY=24,25,26,27,28,29,30;BYDAY=SA
END:STANDARD
END:VTIMEZONE
END:VCALENDAR

These RRULEs essentially mean the last Saturday of the month (as long as 
its not the last day of the month) at 22:00 and 23:00 respectively.

> Formerly I dimly recall there being others with time-of-day greater 
> than 24:00, but unfortunately I don't remember the details. It was 
> probably near the International Date Line. 

It would be handled similar to the above.  The RRULE would model the 
following day with the transition on/after 00:00.

> Plus, we used to have a permanent-DST zone, but I expect iCalendar can 
> handle that (even though POSIX cannot).

Yes, not a problem in iCalendar.

>
> While we're on the topic, I assume that iCalendar can handle Irish 
> time even though it uses negative DST. This is 
> TZ='IST-1GMT0,M10.5.0,M3.5.0/1'. It's not an extension to POSIX; it is 
> pure POSIX (though some software has trouble with it).

BEGIN:STANDARD
TZNAME:IST
TZOFFSETFROM:+0000
TZOFFSETTO:+0100
DTSTART;X-OBSERVED-AT=Z:19810329T010000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:STANDARD
BEGIN:DAYLIGHT

TZNAME:GMT
TZOFFSETFROM:+0100
TZOFFSETTO:+0000
DTSTART;X-OBSERVED-AT=Z:19961027T020000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:DAYLIGHT
END:VTIMEZONE
END:VCALENDAR

-- 
Ken Murchison
Cyrus Development Team
FastMail US LLC

---

From: Paul Eggert <[email-masked]>
Date: Fri, 3 Aug 2018 23:50:11 -0700

Ken Murchison wrote:

> These RRULEs essentially mean the last Saturday of the month (as long as its not 
> the last day of the month) at 22:00 and 23:00 respectively.

What about a TZ value like this?

TZ='<-03>3<-02>,M3.1.0/-2,M10.1.0/-1'

This means to switch forward at 22:00 the day before March's first Sunday, which 
might be February 28 or 29. As I vaguely recall there have been historical rules 
like that, though none are currently active. Similarly for transitions that 
might be at the end of this month or the start of the next.

---

From: Ken Murchison <[email-masked]>
Date: Sat, 4 Aug 2018 09:04:45 -0400

On 08/04/2018 02:50 AM, Paul Eggert wrote:
> Ken Murchison wrote:
>
>> These RRULEs essentially mean the last Saturday of the month (as long 
>> as its not the last day of the month) at 22:00 and 23:00 respectively.
>
> This means to switch forward at 22:00 the day before March's first 
> Sunday, which might be February 28 or 29. As I vaguely recall there 
> have been historical rules like that, though none are currently 
> active. Similarly for transitions that might be at the end of this 
> month or the start of the next.

The iCalendar RRULEs are generated from the Rules in the raw tzdata.  If 
you can give me an example Rule that mimics this, I can run vzic on a 
fake TZ and see what it outputs.  But my guess is that it will look 
something like the following, if starting in 2015:

BEGIN:DAYLIGHT
TZNAME:-02
TZOFFSETFROM:-0300
TZOFFSETTO:-0200
DTSTART;X-OBSERVED-AT=Z:20150228T220000
RRULE:FREQ=YEARLY;BYYEARDAY=-301,-302,-303,-304,-305,-306,-307;BYDAY=SA
END:DAYLIGHT

BEGIN:STANDARD
TZNAME:-03
TZOFFSETFROM:-0200
TZOFFSETTO:-0300
DTSTART;X-OBSERVED-AT=Z:2015100310T230000
RRULE:FREQ=YEARLY;BYYEARDAY=-87,-88,-89,-90,-91,-92,-93;BYDAY=SA
END:STANDARD
END:VTIMEZONE

Negative BYYEARDAY counts backwards from Dec 31, and there are always a 
fixed number of days after the last day of Feb, regardless of leap 
year.  If my math is correct, day -306 is Mar 1, and -307 is the last 
day of Feb.

-- 
Ken Murchison
Cyrus Development Team
FastMail US LLC

---

From: Ken Murchison <[email-masked]>
Date: Sat, 4 Aug 2018 09:48:06 -0400

On 08/03/2018 09:15 PM, Paul Eggert wrote:
> TZ='<-03>3<-02>,M3.5.0/-2,M10.5.0/-1'. Formerly I dimly recall there 
> being others with time-of-day greater than 24:00, but unfortunately I 
> don't remember the details.

Found one: Asia/Jerusalem

IST-2IDT,M3.4.4/26,M10.5.0

BEGIN:DAYLIGHT
TZNAME:IDT
TZOFFSETFROM:+0200
TZOFFSETTO:+0300
DTSTART:20130329T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYMONTHDAY=23,24,25,26,27,28,29;BYDAY=FR
END:DAYLIGHT

-- 
Ken Murchison
Cyrus Development Team
FastMail US LLC

---

From: Paul Eggert <[email-masked]>
Date: Sat, 4 Aug 2018 17:05:58 -0700

Ken Murchison wrote:
> If you can give me an example Rule that mimics this, I can run vzic on a fake TZ 
> and see what it outputs.

If you think it'll work, that's good enough for me. Thanks.