Debian 14 cracks down on unreproducible packages About halfway through the Debian 14 “Forky” development process, its release team announced a new goal: deterministic package compilation. The Debian project’s latest Bits from the release team newsletter has a goal which may not sound very big, but will mean significant extra effort in a direction that could prove to be a …
"In other words, if you use the same version of the same compiler with the same options, then every time you compile an identical set of source files, the process ought to result in an identical set of binary files."
I had implicitly assumed that this was already the case, and I'm moderately shocked to find out it's not.
It always should have been that way, but one of Huawei's many piss-poor coding problems was that the agency in charge of verifying software often could not reproduce the shipped binaries. The obvious conspiracy reason is they had been backdoor-ed, but the Hanlon's razor answer is they did not have proper (maybe any) controls over the build-chain, not the source code.
Also as soon as someone puts a printf("built on %s",BUILD_DATE) or similar, this breaks.
It's also fairly common for developers (and even some toolchains) to encode some build-time system-environment data that is not necessarily reproducible on another system, and one bit difference will completely change the hash of the output file
Essentially, `gcc -D` (i.e. define a preprocessor literal on the commandline) is now 'considered harmful', except that it may be (and very often is, because it is undoubtedly a useful feature) used by pretty much every build system, including CMake etc
It's been a common issue in "serious software engineering" for almost as long as there has been "serious" software (avionics etc) or at least as long as there have been software change control processes
Also as soon as someone puts a printf("built on %s",BUILD_DATE) or similar, this breaks.
If comparing hashes yes. If you do a binary diff and only find a change in an obvious string like that you have less to worry about.
p.s. You can use the __DATE__ pre-processor directive as well.
> p.s. You can use the __DATE__ pre-processor directive as well.
Well, indeed, but AIUI the core problem is precisely things like this: compiler directives passed on the command line, and info the compiler picks up from environment variables, get incorporated into the binary... and as a result, 2 successive builds on the same machine on the same software produce very slightly different outputs.
Essentially, it is. If you put exactly the same bytes through the compilers we're generally using in all the places the compiler looks for bytes, the output is identical. The problem with reproducible builds is that there are a lot of those possible places that people don't always script out. Stuff uses manually-set data, files which the builder is expected to obtain and provide on their own, and all that makes it impossible to reproduce the build unless you know what extra steps the builder took.
For example, I've got an open source project. I publish binaries for it. The Windows one recently updated the Python version it was compiled against while the Mac OS one didn't because I happened to update the version on the VM on which I built the Windows version. I hadn't scripted the "build a Windows package of this" process, so what you get depends on what I did, including possible mistakes or, as in that case, changes which work just fine and people don't notice but don't make it simple for someone else to do exactly what I did without reverse-engineering my actions.
I'll be waiting a little longer to reinstall this box with Debian 14 after its initial release this time than I did with Debian 13.
I'm not budging until the full NVidia chain I need for my RTX4070Ti is ready and supporting Debian 14 natively, not through workarounds and hacks to get Debian 14 to run with Debian 13 repos like I did in earlier attempts to get Debian 13 running on this hardware.
Can such workarounds be effective? Well, you can get an unstable box running that way...
My own .NET Framework applications usually get 0-2 false (i assume :) ) positives on VirusTotal. But sometimes half the vendors flags an application. When I then go and rebuild with AFAICT the only change being the embedded build timestamp, and retest on VT, it is back to 0-2 positives.
More ranting: The behavior page. 2/3 of complaints are things the framework does or the compiler did and I assume does for all applications, or not relevant (I think) like 'Scripting'. Yes it reads the commandline, no need to warn about that. Or 'Registers a vectored exception handler (VEH), possibly to hijack execution flow'. Not entirely sure, but I think that is me catching unhandled exceptions for logging before exit. Looks scary to the customer.
