[tex-k] SORCE_DATE_EPOCH_TEX_PRIMITIVES

[Alexis Bienvenüe]

Hi.

The support of SOURCE_DATE_EPOCH added in pdftex/dvipdfm-x is great, and
does a very good job at building reproducible binary software packages.
SOURCE_DATE_EPOCH allows to fix the metadata timestamps in the compiled
document, and when SOURCE_DATE_EPOCH_TEX_PRIMITIVES is set to 1, the tex
primitives \year, \month etc. are also fixed, so that for example \today
refers to the date given by SOURCE_DATE_EPOCH.

The SOURCE_DATE_EPOCH environment variable is always used to get
reproducible builds, and in this context,
SOURCE_DATE_EPOCH_TEX_PRIMITIVES is always set to 1. This is fine when
working with tex documents only. However in a more general context, for
example debian reproducible build effort [1] (which has equivalents for
other distributions [2]), the need to set another tool-specific
environment variable has been criticized: adding such tool-specifc
envvar handling in general package-building tools is considered to be
endless and so discarded.

Therefore, I would like to support a solution where the default value of
SOURCE_DATE_EPOCH_TEX_PRIMITIVES is set to to 1. It has already been
mentioned by Norman Gray [3]. The only drawback is that we brake
backward-compatibility in a situation where someone already uses
SOURCE_DATE_EPOCH for another task, and don't set
SOURCE_DATE_EPOCH_TEX_PRIMITIVES. In this situation, the default value 0
of SOURCE_DATE_EPOCH_TEX_PRIMITIVES allows this user to get the same
content in the compiled document. I do think this situation is very
unlikely.

Another solution is to drop SOURCE_DATE_EPOCH_TEX_PRIMITIVES as if it is
always set to 1, but I understand that it has to be kept if someone
thinks it can be useful.

Whatever will be your answer, I thank you again for your welcome
regarding reproducibility questions.

Regards,
Alexis Bienvenüe.

[1] https://wiki.debian.org/ReproducibleBuilds/
[2] https://reproducible-builds.org/
[3] https://www.tug.org/pipermail/tex-k/2016-May/002703.html

[Ximin Luo]

(I ask that everyone spend some undisturbed and unpressured time to read and digest what I'm about to write. Hopefully this way we can avoid unnecessary redundant replies that don't contribute new information and only risk adding fuel to any potential flamewars.)

Post by Alexis Bienvenüe
Therefore, I would like to support a solution where the default value of
SOURCE_DATE_EPOCH_TEX_PRIMITIVES is set to to 1.

If I understand correctly, tex-k are pointing out that there is a semantic difference between:

1. implicit timestamps in .tar/.pdf that nobody explicitly told tar/pdf to write
2. explicit calls to time()-equivalents in Turing-complete languages, such as \today in TeX

Yes, we did not explore this issue in enough depth when we originally wrote the SOURCE_DATE_EPOCH spec. Indeed, #2 is exactly why we don't simply patch time(2) ourselves in glibc and be done with it - it does indeed break some use-cases of Turing-complete languages. For example it breaks make(1) which is a very important part of our build ecosystem. I can understand being cautious, when you are parsing a Turing-complete language where #2 *might* have unsafe corner-cases; you don't want to potentially break things that are time-sensitive.

However, in the scenarios that are relevant to PDF generators, we at R-B believe that #2 is *always safe* . Even if your input-language is Turing-complete, whenever you are generating some output data from some input data in a build process, *this should not be time-sensitive or require a "real" clock*. In other words, even if TeX-as-a-general-programming-language may need to be cautious about how it interprets \today, in the context of parsing-a-TeX-file-to-generate-PDFs, this cautiousness is unnecessary and leads to overcomplexity.

Based on this reasoning, our current preferences for pdftex behaviour are:

# Proposal 0 ("ideal" solution for the R-B team)

Use SOURCE_DATE_EPOCH unconditionally for case #2 [*], for the reason in the previous paragraph - i.e. that pdftex will never need to interpret \today in a time-sensitive context, even if TeX in general might need to do that.

[*] i.e. as well as case #1 which is uncontroversial

# Proposal 1

Default SOURCE_DATE_EPOCH_TEX_PRIMITIVES to 1 as Alexis suggested, so that case #2 defaults to honouring SOURCE_DATE_EPOCH, and people doing reproducible builds don't need to learn a new flag/envvar for every new tool.

To allow time-sensitive TeX programs to safely handle corner cases arising out of case #2, you can do it like this:

a. the tex interpreter libraries could have an internal "use-SDE-as-\today" option that defaults to being "no"
b. the pdftex *program* knows that it is using TeX purely in a document-building context which is *not* time-sensitive, and can set this internal option as "yes" by default, even if SOURCE_DATE_EPOCH_TEX_PRIMITIVES is not present. If the user set SOURCE_DATE_EPOCH_TEX_PRIMITIVES=0 explicitly, then it can set the internal option back to "no".

# Proposal 2

If you are unwilling to have case #2 above use SOURCE_DATE_EPOCH by default, then my suggestion is to rename SOURCE_DATE_EPOCH_TEX_PRIMITIVES to a more generic name like USE_CLOCK_SOURCE_DATE.

This is just my personal preference and we the R-B team haven't reached consensus on this yet. My motivation for this name is that it mirrors the POSIX clock id constants CLOCK_REALTIME, CLOCK_MONOTONIC etc. You can read it colloquially as "use CLOCK_SOURCE_DATE [instead of CLOCK_REALTIME]", where CLOCK_SOURCE_DATE can be thought of as a fake constant clock that always returns SOURCE_DATE_EPOCH. (Of course this is not actually in the POSIX standard, but the other clocks are.)

For more context see `man 2 clock_gettime` or http://pubs.opengroup.org/onlinepubs/009695399/functions/clock_getres.html.

X

[Karl Berry]

rename SOURCE_DATE_EPOCH_TEX_PRIMITIVES to a more generic name like
USE_CLOCK_SOURCE_DATE.

I'm fine with that, or any other reasonable name for the second envvar.
Just tell me the name you want.

I don't want to give up the second envvar, or default it to 1, for the
reasons I already gave, with the preferred solutions I already gave.
I understand your point of view but I simply disagree. Sorry ... --karl

[Ximin Luo]

Post by Ximin Luo
rename SOURCE_DATE_EPOCH_TEX_PRIMITIVES to a more generic name like
USE_CLOCK_SOURCE_DATE.
I'm fine with that, or any other reasonable name for the second envvar.
Just tell me the name you want.

OK, we've talked a bit more about it and gotten consensus. How does FORCE_SOURCE_DATE sound?

Ximin

[Karl Berry]

How does FORCE_SOURCE_DATE sound?

Fine by me. Shall I change the pdftex source?

I don't much want to make a new release just for this, but it would be
fine (in fact preferable) to me if you made the change in Debian now.
Since after all the whole thing is for your purposes, you might as well
have the name you want. I doubt the discrepancy will cause any trouble.

Best,
Karl

[Ximin Luo]

Post by Ximin Luo
How does FORCE_SOURCE_DATE sound?
Fine by me. Shall I change the pdftex source?
I don't much want to make a new release just for this, but it would be
fine (in fact preferable) to me if you made the change in Debian now.
Since after all the whole thing is for your purposes, you might as well
have the name you want. I doubt the discrepancy will cause any trouble.

OK, yes we can patch Debian like this. Though, please also change the pdftex source in version control.

Hopefully the next release isn't *too* far ahead - Debian is doing this in most depth at the moment, but other distributions are also following suit - FreeBSD, LEDE, SUSE and Guix already added support for SOURCE_DATE_EPOCH, and we know there's people working on it for Fedora and NixOS.

X

[Norbert Preining]

Hi Ximin,

Post by Ximin Luo
OK, yes we can patch Debian like this. Though, please also change the pdftex source in version control.

Thanks, yes, *I* can patch the sources ;-)

Post by Ximin Luo
Hopefully the next release isn't *too* far ahead

The next release of TL is in *1* year.

Norbert

--
PREINING Norbert + TeX Live & Debian Developer + http://www.preining.info
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13

[Norbert Preining]

Hi Karl,

Post by Ximin Luo
OK, yes we can patch Debian like this. Though, please also change the pdftex source in version control.

Here is a patch of the rename.

Norbert

--
PREINING Norbert + TeX Live & Debian Developer + http://www.preining.info
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13

[Karl Berry]

SOURCE_DATE_EPOCH_TEX_PRIMITIVES -> FORCE_SOURCE_DATE

I changed the sources in the pdftex and tex live repositories for this.
(Thanks Norbert.) -k

[Holger Levsen]

Post by Karl Berry
SOURCE_DATE_EPOCH_TEX_PRIMITIVES -> FORCE_SOURCE_DATE
I changed the sources in the pdftex and tex live repositories for this.
(Thanks Norbert.) -k

yay!! thanks!

[Norbert Preining]

Debian git is already updated, I'll upload sooner or later.... Are we in hurry?
--
PREINING Norbert + TeX Live & Debian Developer + http://www.preining.info
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13

[Holger Levsen]

Post by Norbert Preining
Debian git is already updated, I'll upload sooner or later.... Are we in hurry?

no hurry. (but please do upload "soon", just not with hurry :)