After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 751984 - PATCH: Honour SOURCE_DATE_EPOCH environment variable for reproducible output
PATCH: Honour SOURCE_DATE_EPOCH environment variable for reproducible output
Status: RESOLVED FIXED
Product: doxygen
Classification: Other
Component: general
1.8.11-GIT
Other Linux
: Normal enhancement
: ---
Assigned To: Dimitri van Heesch
Dimitri van Heesch
Depends on:
Blocks:
 
 
Reported: 2015-07-05 13:06 UTC by marivalenm
Modified: 2016-05-09 21:01 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Honour SOURCE_DATE_EPOCH environment variable for reproducible output (2.39 KB, patch)
2015-07-05 13:06 UTC, marivalenm
needs-work Details | Review
Produce timezone independent timestamps to have reproducible output (488 bytes, patch)
2015-08-12 07:08 UTC, marivalenm
none Details | Review
Use UTC timezone when displaying QDateTimes parsed from SOURCE_DATE_EPOCH (1.55 KB, patch)
2016-05-09 20:51 UTC, Infinity Zero
none Details | Review
Use UTC timezone when displaying QDateTimes parsed from SOURCE_DATE_EPOCH (1.51 KB, patch)
2016-05-09 21:01 UTC, Infinity Zero
none Details | Review

Description marivalenm 2015-07-05 13:06:59 UTC
Created attachment 306856 [details] [review]
Honour SOURCE_DATE_EPOCH environment variable for reproducible output

In Debian there are 178 source packages that are not byte for byte reproducible between different rebuilds because doxygen includes a timestamp in its HTML, LaTeX, and Man page output [1] and [2].

Many packages became immediately reproducible after Doxygen changed its HTML_TIMESTAMP default from YES to NO but this still left many packages unreproducible because of two reasons. One is that this change was only implemented for the HTML output but not for the LaTeX or Man page output, the other reason is that some upstream prefer to keep timestamps in their documentation so Debian has to carry a patch for these packages forever.

Attached patch fixes both problems, firstly it applies to the HTML, LaTeX, and Man page output at the same time and secondly it allows for packages to still carry timestamps while being reproducible. The patch works by setting a deterministic timestamps through the environment variable SOURCE_DATE_EPOCH which contains the unix epoch as an integer. This environment variable was introduced by the Debian reproducible builds team but it is meant to be used by any distribution [3]. The package help2man is now supporting this [4] we are also persuading txt2man [5],  epydoc [6], and GCC [7] to support this. 


[1] https://reproducible.debian.net/issues/timestamps_in_documentation_generated_by_doxygen_issue.html
[2] https://reproducible.debian.net/issues/unstable/timestamps_in_manpages_generated_by_doxygen_issue.html
[3] https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal
[4] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=787444
[5] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=790801
[6] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=790899
[7] https://gcc.gnu.org/ml/gcc-patches/2015-06/msg02210.html
Comment 1 Dimitri van Heesch 2015-07-05 13:25:23 UTC
Review of attachment 306856 [details] [review]:

Why is the variable read as a 64 bit int, and then checked for being no bigger than 32 bit?
Will this not cause a "year 2038" problem later on?
Comment 2 marivalenm 2015-07-05 13:39:55 UTC
(In reply to Dimitri van Heesch from comment #1)
> Review of attachment 306856 [details] [review] [review]:
> 
> Why is the variable read as a 64 bit int, and then checked for being no
> bigger than 32 bit?

it is to prepare for when QT does not suffer from the 2038 problem and it also allows for a more precise error reporting because it is used to check if the value of the variable is  within the range of what the QT function can parse and to check if the parsing to an unsigned long long was successful. 

> Will this not cause a "year 2038" problem later on?

Yes Qt currently suffers from the year 2038 problem and there seems to be no function that takes more than an unsigned integer.
Comment 3 Dimitri van Heesch 2015-07-19 10:15:15 UTC
Thanks for the explanation. I've just pushed a variant of your patch to GitHub.
https://github.com/doxygen/doxygen/commit/b31266c1076c6284116f17241d9e8aa048f88e60

Please check if it works correctly.
Comment 4 marivalenm 2015-07-23 14:41:40 UTC
Hi

(In reply to Dimitri van Heesch from comment #3)
> Thanks for the explanation. I've just pushed a variant of your patch to
> GitHub.
> https://github.com/doxygen/doxygen/commit/
> b31266c1076c6284116f17241d9e8aa048f88e60
> 
> Please check if it works correctly.

Thank you for the patch, it works fine for a fixed timezone. 
Unfortunately since I submitted my first patch the Debian reproducible builds team decided not to export TZ=UTC for all package builds because that breaks some packages. 

As a result packages printing timestamps like doxygen must now also take care to not include the user timezone and instead print timestamps in UTC (or any other fixed timezone).

I realized the problem is caused by the call to localtime in the function QDateTime::setTime_t, using gmtime instead of localtime fixes the problem.

Thank you for your effort towards producing reproducible documentation,
akira
Comment 5 marivalenm 2015-08-12 07:08:11 UTC
Created attachment 309101 [details] [review]
Produce timezone independent timestamps to have reproducible output

This patch causes doxygen produce timezone independent timestamps (it has already been tested) though it is probably not the best/most correct way to do it. 

Please tell me what you think.

Cheers!
akira
Comment 6 marivalenm 2015-12-01 12:17:33 UTC
Hello,

Pinging this request.

Cheers,
akira
Comment 7 Dimitri van Heesch 2015-12-30 10:20:25 UTC
This bug was previously marked ASSIGNED, which means it should be fixed in
doxygen version 1.8.11. Please verify if this is indeed the case. Reopen the
bug if you think it is not fixed and please include any additional information 
that you think can be relevant (preferably in the form of a self-contained example).
Comment 8 Infinity Zero 2016-05-09 20:41:59 UTC
Hello, this bug was not fixed correctly. According to the SOURCE_DATE_EPOCH spec [1], "Formatting MUST be deferred until runtime if an end user should observe the value in their own locale or timezone."

However, as Maria pointed out earlier, the current doxygen (including 1.8.11) uses localtime in setTime_t, so the output is in the timezone of the build machine running doxygen, and not the timezone of the reader consuming the final output.

To fix this, one should add a setTimeUTC_t function that is the same as setTime_t, but uses gmtime instead of localtime.

[1] https://reproducible-builds.org/specs/source-date-epoch/
Comment 9 Infinity Zero 2016-05-09 20:51:02 UTC
Created attachment 327542 [details] [review]
Use UTC timezone when displaying QDateTimes parsed from SOURCE_DATE_EPOCH
Comment 10 Infinity Zero 2016-05-09 20:51:46 UTC
It looks like I cannot re-open this bug; I suppose you can do that, or shall I create a new one?
Comment 11 Infinity Zero 2016-05-09 21:01:16 UTC
Created attachment 327544 [details] [review]
Use UTC timezone when displaying QDateTimes parsed from SOURCE_DATE_EPOCH

I was a bit hasty with that previous patch; of course we don't need to call gmtime twice if that's all we're using.