GNOME Bugzilla – Bug 683998
Local Time Zone in Windows always UTC
Last modified: 2012-12-18 23:48:09 UTC
Local time in Windows from the GDateTime class is always in UTC. It is because GTimeZone only knows how to get the time zone in Unix systems. To get the time zone in Windows, we need to use a function from Windows API and use paths relative to the installation directory. I will add the patches that I made to fix this. Please review them first before commiting, I still have some questions about the changes I made.
Created attachment 224292 [details] [review] Just an update for a comment An unimportant patch which updates a comment.
Created attachment 224293 [details] [review] Patch to use system data dirs instead of hard-coded path for searching zoneinfo files Use g_get_system_data_dirs to find zoneinfo dir. - Should the last or first system data directory be checked first? (/usr/share is the last one, I guess?) Some classes in GLib uses the reversed list, others uses the list as-is. I don't know if another directory containing the directory zoneinfo (like /usr/share/zoneinfo) should override the default /usr/share/zoneinfo (the one previously used by GLib). - Should I update the docs for g_time_zone_new which have /etc/localtime? Explaining the Windows part might get quite long. Or maybe I could just say `the equivalent in Windows' or something like that.
Created attachment 224294 [details] [review] Patch to use Windows API to get local time zone This will use the Windows API function GetTimeZoneInformation (introduced in Windows 2000, other newer functions introduced in Windows Vista is also available) to get the time zone offset. Only the time zone offset is used. It will also add the difference in offset when in DST. - Should I update the other information in the fake zoneinfo file? (Like the fields name, isdst, etc.) I don't know much about the file format of zoneinfo files.
Created attachment 224438 [details] [review] Patch to correct system timezone directory for Windows This will keep the old defaults aside Windows.
Created attachment 224440 [details] [review] Patch to use Windows API to get local timezone, updated commit message Updated the commit message.
(In reply to comment #3) > Created an attachment (id=224294) [details] [review] > Patch to use Windows API to get local time zone > > This will use the Windows API function GetTimeZoneInformation (introduced in > Windows 2000, other newer functions introduced in Windows Vista is also > available) to get the time zone offset. Only the time zone offset is used. It > will also add the difference in offset when in DST. > > - Should I update the other information in the fake zoneinfo file? (Like the > fields name, isdst, etc.) I don't know much about the file format of zoneinfo > files. Funny, I've been looking at this, too, after working up a fix for bug 631382. Yes, you should fill in as many fields as you can from the TZi. Note that this only solves the new-local part of the problem: One must mine the registry to get TZi for arbitrary time zones. (In reply to comment #4) > Created an attachment (id=224438) [details] [review] > Patch to correct system timezone directory for Windows > > This will keep the old defaults aside Windows. I don't understand what this is for: Is it so that someone can compile the IANA database and include it? Neither MS nor MinGW provide such a file.
(In reply to comment #6) > (In reply to comment #3) > > Created an attachment (id=224294) [details] [review] [details] [review] > > Patch to use Windows API to get local time zone > > > > This will use the Windows API function GetTimeZoneInformation (introduced in > > Windows 2000, other newer functions introduced in Windows Vista is also > > available) to get the time zone offset. Only the time zone offset is used. It > > will also add the difference in offset when in DST. > > > > - Should I update the other information in the fake zoneinfo file? (Like the > > fields name, isdst, etc.) I don't know much about the file format of zoneinfo > > files. > > Funny, I've been looking at this, too, after working up a fix for bug 631382. > > Yes, you should fill in as many fields as you can from the TZi. > > Note that this only solves the new-local part of the problem: One must mine the > registry to get TZi for arbitrary time zones. I also have thought about that, seems like Windows have some hidden API's for getting time zones other than the current one. They are not declared in the Windows API. And seems like Windows Vista have additional information about the current time zone, though I don't think it would be useful since we need to support Windows XP. > (In reply to comment #4) > > Created an attachment (id=224438) [details] [review] [details] [review] > > Patch to correct system timezone directory for Windows > > > > This will keep the old defaults aside Windows. > > I don't understand what this is for: Is it so that someone can compile the IANA > database and include it? Neither MS nor MinGW provide such a file. I added it there so that users could use the IANA database if they want to use the timezone names used by the IANA database. Windows API only allows to get information about the current time zone (at least those that are not hidden).
>> Note that this only solves the new-local part of the problem: One must mine the >> registry to get TZi for arbitrary time zones. > I also have thought about that, seems like Windows have some hidden API's for > getting time zones other than the current one. They are not declared in the > Windows API. And seems like Windows Vista have additional information about the > current time zone, though I don't think it would be useful since we need to > support Windows XP. Whether or not Microsoft has a private API for retrieving Timezone information from the registry isn't important. The information is in the registry and can be pretty easily retrieved using the standard registry API. Doing so is necessary for a complete GTimezone implementation.
Comment on attachment 224438 [details] [review] Patch to correct system timezone directory for Windows Superseded by 224440
Okay, I'll try investigating this thing on Tuesday, I still have some exams.
Do you have a Windows box that uses a language aside English? I want to know which of those registry keys for time zones are actually localized. According to MSDN, the time zone names are localized which I don't think shouldn't be used as time zone identifiers.
I don't, but I know folks who do, so I've asked. However, I'm not sure that it matters. g_date_time_zone_new() gets the "identifier" string and checks whatever timezone database -- in this case the Time Zones registry folder -- for a match. If it's there, great, load the TZi field from it and process it with zone_for_constant_offset(). If not, use the TZi from GetTimeZoneInformation(), mirroring the POSIX behavior. Passing in the right identifier or setting $TZ correctly isn't our problem, or even something we can control. Let's not make g_date_time_zone_new() any longer than it is already. Put the code in new static functions.
I just thought that those registry paths might be localized. If only the keys are localized, we can only return time zones without DST (the path name and the identifier key for each time zone is the same, unless the identifier key is localized, but the identifier for the "DST version" is localized, if my hunch is correct). For example in Wine's system.reg in the WINEPREFIX: [Software\\Microsoft\\Windows NT\\CurrentVersion\\Time Zones\\Singapore Standard Time] 1347240835 "Display"="Asia/Singapore" "Dlt"="Singapore Daylight Time" "Std"="Singapore Standard Time" "TZI"=hex:20,fe,ff,ff,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,\ 00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00 My hunch is that "Singapore Standard Time" in "Software\\Microsoft\\Windows NT\\CurrentVersion\\Time Zones\\Singapore Standard Time" is probably not localized while the values for the keys "Dlt" and "Std" are probably localized. I want to know whether I'm correct, because changing my Wine's language to Japanese or other languages seems to have "Dlt" and "Std" not localized. I don't know if it still applies to real Windows installations. If they are not localized then that would be great; we could compare the identifier passed to g_time_zone_new with both "Dlt" and "Std" keys.
The same key exported on a real XP - nothing of the keys is localized, just some values: Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Time Zones\Singapore Standard Time] "Display"="(GMT+08:00) Kuala Lumpur, Singapur" "Dlt"="Malaysische Halbinsel Sommerzeit" "Std"="Malaysische Halbinsel Normalzeit" "MapID"="16,17" "Index"=dword:000000d7 "TZI"=hex:20,fe,ff,ff,00,00,00,00,c4,ff,ff,ff,00,00,00,00,00,00,00,00,00,00,00,\ 00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00
Good. That matches what my other source tells me. Looks like Wine has changed the TZ Display to the IANA standard from MS's. I wonder what else they've done about timezones. OTOH, they've also discarded what I think is the summer time offset. Anyway, I can see a client program providing an identifier from the IANA list, from the list of registry folder names, or from the localized Display or Std/Dlt strings. It's obviously easy to handle the case where the identifier matches one of the folder names, and only slightly harder to match an IANA timezone name (cross reference in XML here: http://unicode.org/repos/cldr/trunk/common/supplemental/windowsZones.xml). I wonder if it's worth the effort to try to match the localized values inside the registry node, though. That would require searching through each each registry node, either every time or the first time to build a translation hash.
Thanks, John Ralls and Hans Breuer. I'm planning to check only the unlocalized names, which is the path (or folder name). Maybe I'll check if it has "Daylight" in its name, then replace it with "Standard", taking note that I will use the DST setting. Do you think we should include some code to check the IANA values in http://unicode.org/repos/cldr/trunk/common/supplemental/windowsZones.xml? Maybe we could add this as a GResource?
Created attachment 226225 [details] [review] Patch to use Windows API to get time zone from registry I edited the patch to get the time zones from the registry. GetTimeZoneInformation() is used to get the local timezone. I don't know how to set the transition times for daylight, so it will just check if the identifier has "Daylight" in its name then give the timezone with the daylight or standard interval (i.e. the time zones for standard and daylight are separated instead of in a single GTimeZone). I think I should put two intervals if daylight date is available, but I don't know much about the file format of tzfile. I can't find a good reference that explains it fully. Also the data types of structures in gtimezone.c and from the man page tzfile doesn't match. Is it because they are using different versions of the file format.
Good start, but there's a fair amount of duplication there: the fake tzfile headers, for example. Also, I realize that there are indentation errors in the existing file, but please don't include fixing them in this patch. Keep it focused on the windows timezones. Same with the /usr/share/timezone|zoneinfo comment. If you really want, attach those other changes as a separate patch and I'll commit it for you. Standard time and Daylight time are a single timezone selected by "interval", and GDateTime (which works closely with GTimeZone) needs to be able to select the appropriate interval in order to return the correct time in a timezone. Creating different timezones for standard and daylight time will break that. There's a wee note at the end of tzfile(5) about how in the TZif2 format the data are duplicated after the TZif section with 64-bit values, and that's what gtimezone.c (at least until I push my patch from bug 631382, which will use the TZif structure if TZif2 one isn't available) is looking at. That probably explains what you're seeing as type mismatches. I've checked the file formats in a hex editor and tzfile(5) is correct and complete, so you can use it.
Created attachment 226228 [details] [review] Patch to use Windows API to get time zone from registry, updated I merged standard and daylight into one time zone. Daylight will no longer be checked, the Standard time zone names should be used. Identifier of each interval (standard and daylight) is offset in string. Made a single function to create a fake tzfile for bias. Only bias'es are used in TIME_ZONE_INFORMATION and registry. Removed some parts of the patch that were part of fix for Windows path to database (those indentation fixes and other things related). Updated documentation for g_time_zone_new(). Seems like your implementation is correct, the man pages shows incorrect data types in the declarations, but the explanations show the true length (i.e. says int in tt_isdst but explains that it's a single byte only).
Should we map IANA time zone names to Microsoft time zone names?
Forgot to mention that the time zones will always have a daylight interval even if the time zone doesn't support it. Should I fix that?
Created attachment 226230 [details] [review] Indentation and comment fixes If you want you may also commit this.
Comment on attachment 226230 [details] [review] Indentation and comment fixes Pushed to master and glib-2-34.
Created attachment 226364 [details] [review] Patch to use Windows API to get time zone from registry, update 2 Create one interval only when daylight is not supported. Fix index of abbreviations. Add 2 transition times when daylight is supported, pointing to standard and daylight. Transition times are both 0.
Created attachment 226365 [details] Test for GTimeZone I tried testing GTimeZone using this source. The outputs seems the same (I used IANA equivalents for Unix), but getting interval info in Linux produces segmentation faults. I Use Ubuntu Linux 12.10, the version of libglib2.0-0 is 2.34.0-1ubuntu1. Output from Wine: ================================================================================ *** Converting ISO-8601 Date to Local Time Zone and UTC *** Date in ISO8601 format: 2012-09-09T01:30:00Z Date parsed by g_time_val_from_iso8601: 1347154200 0 Date converted by g_date_time_new_from_timeval_local: 2012-09-09T09:30:00 Date converted by g_date_time_new_from_timeval_utc: 2012-09-09T01:30:00Z *** Converting ISO-8601 Date to Different Time Zones *** Date converted to Singapore Standard Time: 2012-09-09T09:30:00 Abbreviation: +0800 Is DST: No Date converted to Tokyo Standard Time: 2012-09-09T10:30:00 Abbreviation: +0900 Is DST: No Date converted to Pacific Standard Time: 2012-09-08T18:30:00 Interval: 0 Abbreviation: -0800 Is DST: No Interval: 1 Abbreviation: -0800 Is DST: No Interval: 2 Abbreviation: -0700 Is DST: Yes ================================================================================ To compile: "i686-w64-mingw32-gcc -g time-check.c -o time-check.exe `PKG_CONFIG_LIBDIR= PKG_CONFIG_PATH=/home/kyoushuu/Devel/MinGW/lib/pkgconfig pkg-config --cflags --libs glib-2.0`" (Cross-compiled GLib is installed in /home/kyoushuu/Devel/MinGW) Output from Linux (without interval info): ================================================================================ *** Converting ISO-8601 Date to Local Time Zone and UTC *** Date in ISO8601 format: 2012-09-09T01:30:00Z Date parsed by g_time_val_from_iso8601: 1347154200 0 Date converted by g_date_time_new_from_timeval_local: 2012-09-09T09:30:00 Date converted by g_date_time_new_from_timeval_utc: 2012-09-09T01:30:00Z *** Converting ISO-8601 Date to Different Time Zones *** Date converted to Singapore Standard Time: 2012-09-09T09:30:00 Date converted to Tokyo Standard Time: 2012-09-09T10:30:00 Date converted to Pacific Standard Time: 2012-09-08T18:30:00 ================================================================================ To compile: "gcc -g time-check.c -o time-check `pkg-config --cflags --libs glib-2.0`" Output from Linux (with interval info): ================================================================================ *** Converting ISO-8601 Date to Local Time Zone and UTC *** Date in ISO8601 format: 2012-09-09T01:30:00Z Date parsed by g_time_val_from_iso8601: 1347154200 0 Date converted by g_date_time_new_from_timeval_local: 2012-09-09T09:30:00 Date converted by g_date_time_new_from_timeval_utc: 2012-09-09T01:30:00Z *** Converting ISO-8601 Date to Different Time Zones *** Date converted to Singapore Standard Time: 2012-09-09T09:30:00 Segmentation fault (core dumped) ================================================================================ To compile: "gcc -g time-check.c -o time-check `pkg-config --cflags --libs glib-2.0` -DSHOW_INTERVALS" In GDB: ================================================================================ GNU gdb (GDB) 7.5-ubuntu Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /home/kyoushuu/Devel/MinGW/time-check...done. (gdb) r Starting program: /home/kyoushuu/Devel/MinGW/time-check [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". *** Converting ISO-8601 Date to Local Time Zone and UTC *** Date in ISO8601 format: 2012-09-09T01:30:00Z Date parsed by g_time_val_from_iso8601: 1347154200 0 Date converted by g_date_time_new_from_timeval_local: 2012-09-09T09:30:00 Date converted by g_date_time_new_from_timeval_utc: 2012-09-09T01:30:00Z *** Converting ISO-8601 Date to Different Time Zones *** Date converted to Singapore Standard Time: 2012-09-09T09:30:00 Program received signal SIGSEGV, Segmentation fault. g_time_zone_get_abbreviation (tz=0x6048c0, interval=<optimized out>) at /build/buildd/glib2.0-2.34.0/./glib/gtimezone.c:760 760 return tz->abbrs + interval_abbrind (tz, interval); (gdb) print tz $1 = (GTimeZone *) 0x6048c0 (gdb) bt
+ Trace 231029
$2 = {name = 0x604840 "", zoneinfo = 0x0, header = 0x7ffff7fef0ab, infos = 0x7ffff7fef128, trans = 0x7ffff7fef0d7, indices = 0x7ffff7fef11f <Address 0x7ffff7fef11f out of bounds>, abbrs = 0x7ffff7fef15e <Address 0x7ffff7fef15e out of bounds>, timecnt = 9, ref_count = 1} (gdb) q A debugging session is active. Inferior 1 [process 30305] will be killed. Quit anyway? (y or n) y ================================================================================
Forgot to mention that the one installed in Linux is from Ubuntu. I didn't compile it from source. Also I use 64-bit Linux and 32-bit Wine. Seems like both Linux and Windows uses the one with daylight. In my patch it is because both standard and daylight has 0 transition times.
Do I have to compute all transition times, every year that should be supported? I tried using zic and it will compute all years using the date in DST. I tried something like: ================================================================================ # Rule NAME FROM TO TYPE IN ON AT SAVE LETTER/S Rule Phil minimum maximum - Apr 1 0:00 1:00 S Rule Phil minimum maximum - June 1 0:00 0:00 - # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone Asia/Manila 8:00 Phil PH%sT ================================================================================ And the resulting file has 276 transition times with 2 local time types.
I'm planning to fix GTimeZone to include other TZ formats first, as explained in "man tzname", since the transition times calculated there and in Windows are almost same. Should I file another bug report afterwards?
> Do I have to compute all transition times, every year that should be supported? I don't think so. The IANA Singapore file has only 8 transitions. > I'm planning to fix GTimeZone to include other TZ formats first, as explained > in "man tzname", since the transition times calculated there and in Windows are > almost same. Sorry, I don't understand. Do you mean that you'll look for an IANA zoneinfo directory on Win32 before consulting the registry? > but getting interval info in Linux produces segmentation faults I can replicate that in a git-master build on OSX. I'm investigating.
Hmm. When I run time-check under Win7Pro I get some asserts: Interval: 1 (time-check.exe:4084): Glib-CRITICAL **: g_time_zone_get_abbreviation: assertion `interval_valid (tz, interval)' failed Abbreviation: (null) (time-check.exe:4084): Glib-CRITICAL **: g_time_zone_is_dst: assertion `interval_valid (tz, interval)' failed Is DST: No Interval: 2 (time-check.exe:4084): Glib-CRITICAL **: g_time_zone_get_abbreviation: assertion `interval_valid (tz, interval)' failed Abbreviation: (null) (time-check.exe:4084): Glib-CRITICAL **: g_time_zone_is_dst: assertion `interval_valid (tz, interval)' failed Is DST: No
(In reply to comment #29) > > Do I have to compute all transition times, every year that should be supported? > > I don't think so. The IANA Singapore file has only 8 transitions. > Probably because Singapore doesn't support DST completely. In the Philippines we don't use DST with some dates (lasted only a day each) having DST. I think this would be necessary for intervals. > > I'm planning to fix GTimeZone to include other TZ formats first, as explained > > in "man tzname", since the transition times calculated there and in Windows are > > almost same. > > Sorry, I don't understand. Do you mean that you'll look for an IANA zoneinfo > directory on Win32 before consulting the registry? > I mean I would be fixing parsing of the environment variable TZ first. TZ works the same with Windows' time zones: offsets and transition dates are specified for standard and daylight. The transition dates differs every year, but rules are specified on how to compute them. See the Linux man page for tzset (Mm.w.d and Jn rules) and TIME_ZONE_INFORMATION (StandardDate and DaylightDate) in MSDN. This will minimize the code because I could reuse them to compute each transition time. I'm almost finish in this part. The sample TZ for New Zealand works. I haven't tested it in Linux yet. > > but getting interval info in Linux produces segmentation faults > > I can replicate that in a git-master build on OSX. I'm investigating.
(In reply to comment #30) > Hmm. When I run time-check under Win7Pro I get some asserts: > Interval: 1 > > (time-check.exe:4084): Glib-CRITICAL **: g_time_zone_get_abbreviation: > assertion `interval_valid (tz, interval)' failed > Abbreviation: (null) > > (time-check.exe:4084): Glib-CRITICAL **: g_time_zone_is_dst: assertion > `interval_valid (tz, interval)' failed > Is DST: No > > Interval: 2 > > (time-check.exe:4084): Glib-CRITICAL **: g_time_zone_get_abbreviation: > assertion `interval_valid (tz, interval)' failed > Abbreviation: (null) > > (time-check.exe:4084): Glib-CRITICAL **: g_time_zone_is_dst: assertion > `interval_valid (tz, interval)' failed > Is DST: No I try it again after fixing the patch for TZ parser. Do not have any problems in Wine though. I still haven't tested the latest patch in Windows 7 and XP.
Created attachment 226392 [details] [review] Patch for parsing TZ Here is the current patch for parsing TZ. You have to remove the patch for Windows' time zone because I still haven't fixed it to apply above this patch. This currently only supports the Unix formats for TZ envvar. I'm planning to add MS' format, but it lacks some features (daylight offset is fixed to +1 hour of standard, it is from local to UTC too (negation of Unix's offset), the dates are for United States for all time zones...). So I'm hesitating to use it. What do you think? I probably would have used Unix formats for Windows too, but negation of the offset might confuse users. Also, mktime in Windows is problematic: you cannot use it with pre-1970 dates and limited to 2037, while in Linux you are allowed from 1900 to 2038. Files in /usr/share/zoneinfo usually contains intervals from 1900 to 2038. I'll try looking for an alternative; maybe using GDate to find the weekday and to get the transition time would be better. I included some g_printf so you could see the generated intervals. You could try "NZST-12:00:00NZDT-13:00:00,M10.1.0,M3.3.0" (New Zealand, from tzset manpage). tzset in Linux: http://linux.die.net/man/5/tzfile tzset in Windows: http://msdn.microsoft.com/en-US/library/90s5c885%28v=vs.80%29.aspx For the Windows time zone from registry, should I map them with their IANA equivalents and abbreviations so that IANA names could be used in Windows too? I'll probably use a hash table to lookup the equivalent MS time zones as values with IANA time zones as keys.
Well, I found the source of your crashes: You're calling g_time_zone_unref(tz) before trying to get the abbreviation and is_dst. That should cause problems with Win32 as well, but at present it seems that I'm not linking with the right glib dll, so it's not working at all, of course.
*** Bug 654347 has been marked as a duplicate of this bug. ***
Comment on attachment 226392 [details] [review] Patch for parsing TZ ISTM this is out-of-scope for this bug. mktime and friends are problematic everywhere, and you mustn't use them here at all: The point of GDateTime and GTimeZone is to have a libc-independent date/time implementation. So, at the minimum, get rid of mktime and open a new bug. But ISTM that if GTimeZone is going to support arbitrary timezone creation we need to re-think the "fake tzif" approach.
(In reply to comment #34) > Well, I found the source of your crashes: You're calling g_time_zone_unref(tz) > before trying to get the abbreviation and is_dst. That should cause problems > with Win32 as well, but at present it seems that I'm not linking with the right > glib dll, so it's not working at all, of course. Didn't noticed that. Should I attach a glib binary? I'll be attaching the newer patches (with Windows time zones) next.
Created attachment 226396 [details] [review] Patch to use Windows API to get time zone from registry, update 3 (depends on TZ parsing patch)
(In reply to comment #36) > (From update of attachment 226392 [details] [review]) > ISTM this is out-of-scope for this bug. > > mktime and friends are problematic everywhere, and you mustn't use them here at > all: The point of GDateTime and GTimeZone is to have a libc-independent > date/time implementation. > > So, at the minimum, get rid of mktime and open a new bug. But ISTM that if > GTimeZone is going to support arbitrary timezone creation we need to re-think > the "fake tzif" approach. I'll rewrite it to use GDateTime then. Also I can't think of anything better than the fake tzif one (though it has to recreate about 260+ intervals per time zone with daylight).
Created attachment 226397 [details] Test for GTimeZone
> I'll rewrite it to use GDateTime then. You'll have to be very careful with that: g_date_time_new() takes a GTimeZone, and you're writing code that goes into g_time_zone_new(). > Also I can't think of anything better than the fake tzif one. Rather than creating a fake tzif, create struct _GTimeZone with the necessary components, and a set of new_from functions which populate the struct. > (though it has to recreate about 260+ intervals per time zone with daylight). I don't think that that's right. Even IANA's Los_Angeles file has only 185, and it's a 64-bit file that in theory goes way into the future.
(In reply to comment #41) > > I'll rewrite it to use GDateTime then. > > You'll have to be very careful with that: g_date_time_new() takes a GTimeZone, > and you're writing code that goes into g_time_zone_new(). > I'll use UTC then, which is used as start and end dates in Windows (don't know in Unix, no information in the manpage). Or might make a new data type for a rule stating on how to compute transition dates. > > Also I can't think of anything better than the fake tzif one. > > Rather than creating a fake tzif, create struct _GTimeZone with the necessary > components, and a set of new_from functions which populate the struct. > What about creating a new data type for each interval (transition times) and local time type? Or something like rules, just like in files used by zic to create the tzif files (explained in zic manpage)? Though that might use up more memory and processing power for each time zone in Unix-like systems, and it will be a lot more complex. I'm also planning to support Dynamic DST in Windows. I might create a function that creates a time zone based on rules and transition dates. > > (though it has to recreate about 260+ intervals per time zone with daylight). > > I don't think that that's right. Even IANA's Los_Angeles file has only 185, and > it's a 64-bit file that in theory goes way into the future. That's because different teritories started using DST in different years: some started in 1916, while others mostly started in 1940s. (In http://en.wikipedia.org/wiki/Daylight_saving_time_by_country, about 8 teritories started in 1916 or 1917 while 7 teritories started in 1940s.) Windows and TZ environment variable do not provide this information. Maybe I should just start in 1970? Since that is the start of the Unix epoch, and there should be only a small number of systems that would be storing dates before that year. That would be (2037-1970+1)*2 = 134 intervals.
> I'll use UTC then, Well, yes, of course. The main thing is to avoid getting into an infinite recursion where g_time_zone_new() calls g_date_time_new() calls g_time_zone_new()... > which is used as start and end dates in Windows (don't know > in Unix, no information in the manpage) Read the manpage more closely: Then there are tzh_ttisstdcnt standard/wall indicators, each stored as a one-byte value; they tell whether the transition times associated with local time types were specified as standard time or wall clock time, and are used when a time zone file is used in handling POSIX-style time zone environment variables. Finally there are tzh_ttisgmtcnt UTC/local indicators, each stored as a one-byte value; they tell whether the transition times associated with local time types were specified as UTC or local time, and are used when a time zone file is used in handling POSIX-style time zone environment variables. > What about creating a new data type for each interval (transition times) and local time type? I was thinking of something along this line: typedef struct { gint32 gmt_offset; gboolean is_dst; gboolean is_standard; gboolean is_gmt; gchar *abbrev; } TransitionInfo; typedef struct { gint64 time; gint info_index; } Transition; /* GTimeZone structure and lifecycle {{{1 */ struct _GTimeZone { gchar *name; GArray *t_info; GArray *transitions; gint ref_count; }; We'd then have a set of static functions init_zone_from_foo(GTimeZone*, gpointer) which would take a source (IANA file, Windows TZI, bare offset, POSIX-style TZ string, etc.) and stuff the structures. > That's because different teritories started using DST in different years: > some started in 1916, while others mostly started in 1940s. (In > http://en.wikipedia.org/wiki/Daylight_saving_time_by_country, about 8 > teritories started in 1916 or 1917 while 7 teritories started in 1940s.) > Windows and TZ environment variable do not provide this information. > Maybe I should just start in 1970? Since that is the start of the Unix epoch, > and there should be only a small number of systems that would be storing dates > before that year. That would be (2037-1970+1)*2 = 134 intervals. A bit of probing in the debugger shows that for LA, interval 1 begins 1918-03-31 03:00:00, interval 4 begins 1919-10-26 01:00:00, interval 6 begins 1945-08-14 16:00:00 (that's probably an error in the database), interval 6 and interval 184 begins 2037-03-08 03:00:00. That does account for the missing transitions. Yes, since M$ doesn't support dates before 1970-01-01 00:00:00 Z, you'd be doing it wrong if you projected dst intervals before then. In fact, for Wine/Crossover, which don't have Dynamic DST (Win2K and later *do* have it thanks to updates), you can't really generate correct DST intervals before whenever the current transition dates took effect (2007 in the US). The TZif2 format offers another approach: After the second header and data comes a newline-enclosed, POSIX- TZ-environment-variable-style string for use in handling instants after the last transition time stored in the file (with nothing between the newlines if there is no POSIX representation for such instants). As you noticed, this is largely analogous to the Win32 way of marking transitions. Since even with the current IANA database, interval data ends at 2037, g_time_zone_find_interval() needs to be able to handle generalized transition dates (i.e. M-D H:M:S) anyway, so rather than computing a bunch of fixed intervals á la zic, perhaps it makes more sense to have two types of Transition, a precomputed time_t from the IANA database or a generalized date.
(In reply to comment #43) > > I'll use UTC then, > > Well, yes, of course. The main thing is to avoid getting into an infinite > recursion where g_time_zone_new() calls g_date_time_new() calls > g_time_zone_new()... I unlocked time_zones before using g_date_time_*, is that safe enough? > > > which is used as start and end dates in Windows (don't know > > in Unix, no information in the manpage) > > Read the manpage more closely: > Then there are tzh_ttisstdcnt standard/wall indicators, each stored as a > one-byte value; they tell whether the transition times associated with > local time types were specified as standard time or wall clock time, and > are used when a time zone file is used in handling POSIX-style time zone > environment variables. > > Finally there are tzh_ttisgmtcnt UTC/local indicators, each stored as a > one-byte value; they tell whether the transition times associated with > local time types were specified as UTC or local time, and are used when a > time zone file is used in handling POSIX-style time zone environment > variables. > Didn't notice it, because the fake tzif doesn't use them. I'll try looking at them later. > > > What about creating a new data type for each interval (transition times) and > local time type? the local time type is the tt_info structs in tzif, analogous to rules in zic. > > I was thinking of something along this line: > typedef struct > { > gint32 gmt_offset; > gboolean is_dst; > gboolean is_standard; > gboolean is_gmt; > gchar *abbrev; > } TransitionInfo; > > typedef struct > { > gint64 time; > gint info_index; > } Transition; Here the Transition is called "transition times", TransitionInfo is "local time types". That's what I meant. > > > /* GTimeZone structure and lifecycle {{{1 */ > struct _GTimeZone > { > gchar *name; > GArray *t_info; > GArray *transitions; > gint ref_count; > }; > > We'd then have a set of static functions init_zone_from_foo(GTimeZone*, > gpointer) which would take a source (IANA file, Windows TZI, bare offset, > POSIX-style TZ string, etc.) and stuff the structures. > > > That's because different teritories started using DST in different years: > > some started in 1916, while others mostly started in 1940s. (In > > http://en.wikipedia.org/wiki/Daylight_saving_time_by_country, about 8 > > teritories started in 1916 or 1917 while 7 teritories started in 1940s.) > > Windows and TZ environment variable do not provide this information. > > > Maybe I should just start in 1970? Since that is the start of the Unix epoch, > > and there should be only a small number of systems that would be storing dates > > before that year. That would be (2037-1970+1)*2 = 134 intervals. > > A bit of probing in the debugger shows that for LA, interval 1 begins > 1918-03-31 03:00:00, interval 4 begins 1919-10-26 01:00:00, interval 6 begins > 1945-08-14 16:00:00 (that's probably an error in the database), interval 6 and > interval 184 begins 2037-03-08 03:00:00. That does account for the missing > transitions. I created a simple program to show intervals, based on GTimeZone. I'll attach it here. > > Yes, since M$ doesn't support dates before 1970-01-01 00:00:00 Z, you'd be > doing it wrong if you projected dst intervals before then. In fact, for > Wine/Crossover, which don't have Dynamic DST (Win2K and later *do* have it > thanks to updates), you can't really generate correct DST intervals before > whenever the current transition dates took effect (2007 in the US). I'm planning to fallback to the "Non-dynamic" DST settings if they are not available. > > The TZif2 format offers another approach: > > After the second header and data comes a newline-enclosed, POSIX- > TZ-environment-variable-style string for use in handling instants > after the last transition time stored in the file (with nothing > between the newlines if there is no POSIX representation for such > instants). > > As you noticed, this is largely analogous to the Win32 way of marking > transitions. Since even with the current IANA database, interval data ends at > 2037, g_time_zone_find_interval() needs to be able to handle generalized > transition dates (i.e. M-D H:M:S) anyway, so rather than computing a bunch of > fixed intervals á la zic, perhaps it makes more sense to have two types of > Transition, a precomputed time_t from the IANA database or a generalized date. So, I will handle it too (TZ in TZif2)? I will create a new type of Transition for generalized and precomputed dates?
Created attachment 226434 [details] Get time zone information First argument is database file name ("Asia/Manila"), TZ environment variable value ("PST8PDT" (Windows format, in Windows only), "PST-08:00:00PDT-07:00:00,M4.1.0,M10.5.0" (POSIX format)) or MS time zone name ("Singapore Standard Time").
(In reply to comment #43) > Yes, since M$ doesn't support dates before 1970-01-01 00:00:00 Z, you'd be > doing it wrong if you projected dst intervals before then. In fact, for > Wine/Crossover, which don't have Dynamic DST (Win2K and later *do* have it > thanks to updates), you can't really generate correct DST intervals before > whenever the current transition dates took effect (2007 in the US). > Should I stick to 1900 as start year for Unix?
New related bug bgo686128 opened. We should move discussion of these new structures to the new bug report.
Created attachment 226490 [details] [review] Parse Windows TZ format Apply patches from 686128 before applying this one.
Created attachment 226491 [details] [review] Patch to use Windows API to get time zone from registry, update 4 This depends on the previous patch.
Created attachment 226492 [details] Get time zone information Updated to use local time and zoneinfo version 1 files. Define NO_ZONEINFO_V1 during compile time to use it on builds without the commit 86a8ec0 (GTimeZone support for zoneinfo version 1).
Created attachment 226596 [details] [review] Parse Windows TZ format See Bug 686128 for more info.
Created attachment 226598 [details] [review] Patch to use Windows API to get time zone from registry, update 5 Updated to use new structures. Now supports Dynamic DST. Falls back to old mechanism if not found in database. Dynamic DST is also supported in local time zone, it will check the registry if there's a TimeZoneKeyName in current control set. Didn't tested it much, the test is in Bug 686128. Tested Dynamic DST in Wine by adding Dynamic DST of Pacific Standard Time from http://support.microsoft.com/kb/931836 to the registry.
Created attachment 226879 [details] [review] Parse Windows format for TZ
Created attachment 226880 [details] [review] Fix time zones in Windows
Now using Pacific Standard Time dates for Daylight Savings Time in TZ environment variable, which is probably how Windows works (the default Windows time zone is PST if not set in TZ or registry). Updated documentation. Some changes for rules.
Created attachment 226900 [details] [review] Parse Windows format for TZ Use newly added members isstd and isgmt in rules.
When I set test_GDateTime_equal() at tests/gdatetime.c:192 to this: #ifdef G_OS_UNIX tz = g_time_zone_new ("America/Recife"); #elif defined G_OS_WIN32 tz = g_time_zone_new ("E. South America Standard Time"); #endif I get a crash in init_zone_from_rules() at gtimezone.c:858 gtz->transitions = g_array_sized_new (FALSE, TRUE, sizeof (Transition), trans_count); because trans_count < 0 (which naturally becomes a very large guint in the alloc call). Trans_count is going to zero because for some reason the start time in rule 5 is 0, so (rules[i+1].start_year - rules[i].start_year) becomes -2010. I haven't stepped through everything, but there are apparently more like that, because at the end of that loop, trans_count = -11828. As a first guess, something's not right with your duplicate-rule code in the Dynamic DST section of rules_from_windows_time_zone(). Two things, actually, because it came up with 20 rules, but when I look at it in the registry there are 5 varying rules (2006, 7, 8, 9, and 10), and then 29 identical rules (2010 to 2040), but rules_count was 20. I'll leave it to you to debug further. At the minimum, tests/gdatetime should pass all tests under windows with the possible exception of test_non_utf8_printf(). It would be really good if you could write more tests that exercise your new functionality.
Sorry, closed that by mistake.
(In reply to comment #57) > I'll leave it to you to debug further. At the minimum, tests/gdatetime should > pass all tests under windows with the possible exception of > test_non_utf8_printf(). It would be really good if you could write more tests > that exercise your new functionality. I'm planning to make the tests work in Windows too, but that would require to change the dates to about pre-2003, because Windows XP doesn't have the new rules for time zones in the registry. I also have to compile it in a real Windows installation (I could use a virtual environment, but I don't have much space left in my hard drive), and MinGW is very slow when compiling in Windows, so I'm delaying it a bit. I can't run the tests in Wine either, because I'm cross-compiling.
Found the problem in Comment 57: seems like the indexes when putting the time zone information to the rules is wrong, leaving some to have 0 as starting year, making the loops in init_zone_from_rules to fail. I will attach the patches later. Also, seems like there are Windows time zones that go up to 2040. Should I make the maximum 2037 still, or should I parse them too? I don't know if there's some overflow that might happen, though it seems like the intervals shows up fine in my tests in Wine.
Created attachment 226925 [details] [review] Parse Windows format for TZ Fixed Dynamic DST.
Created attachment 226926 [details] [review] Fix time zones in Windows Renamed GTimeZoneRule and GTimeZoneDate to TimeZoneRule and TimeZoneDate.
(In reply to comment #60) > Also, seems like there are Windows time zones that go up to 2040. Should I make > the maximum 2037 still, or should I parse them too? I can't imagine that there's any difference between the 2038 and 2040 rules, so I don't think that it really matters. OTOH, a couple of extra transitions isn't going to overflow anything. A bigger concern (and this applies to IANA-based timezones as well) is how to make DST work past the end of the transitions, but that's not really germane to this bug.
(In reply to comment #59) > I'm planning to make the tests work in Windows too, but that would require to > change the dates to about pre-2003, because Windows XP doesn't have the new > rules for time zones in the registry. I also have to compile it in a real > Windows installation (I could use a virtual environment, but I don't have much > space left in my hard drive), and MinGW is very slow when compiling in Windows, > so I'm delaying it a bit. I can't run the tests in Wine either, because I'm > cross-compiling. As long as the XP user is keeping up with updates, the TZ data in the registry should be the same between XP and Win7. That will change soon when MS stops updating XP, but even then it will be correct up to that point. If you can't run tests, then you can't finish you patch. Compiling successfully isn't sufficient. Shall I take over at this point? I can start a new commit with fixes, and push when I'm satisfied that everything works and is properly unit tested.
(In reply to comment #64) > (In reply to comment #59) > > I'm planning to make the tests work in Windows too, but that would require to > > change the dates to about pre-2003, because Windows XP doesn't have the new > > rules for time zones in the registry. I also have to compile it in a real > > Windows installation (I could use a virtual environment, but I don't have much > > space left in my hard drive), and MinGW is very slow when compiling in Windows, > > so I'm delaying it a bit. I can't run the tests in Wine either, because I'm > > cross-compiling. > > As long as the XP user is keeping up with updates, the TZ data in the registry > should be the same between XP and Win7. That will change soon when MS stops > updating XP, but even then it will be correct up to that point. > Ah, right. I actually don't update my Windows that's why I said that (because I don't use it much anyway; I only use it for testing my programs). > If you can't run tests, then you can't finish you patch. Compiling successfully > isn't sufficient. Shall I take over at this point? I can start a new commit > with fixes, and push when I'm satisfied that everything works and is properly > unit tested. Yeah, if it's fine with you, you could take over for the unit tests. I don't have experience writing unit tests. If there's a bug and you can't find the problem, you could just tell me. Does gtester works in Windows? Because there is an if OS_UNIX statement in glib/tests/Makefile.am. Only tests/refcount tests are working in Wine because of this.
(In reply to comment #63) > > Also, seems like there are Windows time zones that go up to 2040. Should I make > > the maximum 2037 still, or should I parse them too? > > I can't imagine that there's any difference between the 2038 and 2040 rules, so > I don't think that it really matters. OTOH, a couple of extra transitions isn't > going to overflow anything. I just thought that it may overflow with time_t in 32-bits, though it may not since GLib uses gint64 anyway. > > A bigger concern (and this applies to IANA-based timezones as well) is how to > make DST work past the end of the transitions, but that's not really germane to > this bug. I didn't do it because I can't understand find_interval fully and needs a lot of other changes; I might break something if I can't fully understand some code.
*** Bug 676935 has been marked as a duplicate of this bug. ***
Created attachment 227884 [details] [review] Parse Windows format for TZ Fixed some maybe-uninitialized warnings.