The Unix Epochalypse might be sooner than you think A stark warning about the upcoming Epochalypse, also known as the "Year 2038 problem," has come from the past, as National Museum Of Computing system restorers have discovered an unsetting issue while working on ancient systems. iplayer Epoch-alypse now: BBC iPlayer flaunts 2038 cutoff date, gives infrastructure game away …
This post has been deleted by its author
If I'm not wrong, FAA has systems that dates back to the 1960s. It's not a SCADA issue only. In eleven years, there could be a lot of systems that are older than forty years still running.
And it doesn't matter if the underlying hardware is newer - if the software is not.
What systems will still be in use in 2038 or so, that have been made before 2000?
Loads of them.
There were plenty still in use in 2000 from the 70s and 80s. Computers since then have become so much faster that their useful lifespan has only increased. I'm typing this on a nearly 5 year old PC that I've no intention of replacing any time soon. Family members are using 8 year old machines that are just fine. That's just the computers.
Now think about everything else you use that has some kind of time/date function. 32bit embedded stuff is everywhere. Googling suggests the problem was fixed around the mid/late 2010s, so assuming everything embedded was using up-to-date tooling after that then we're good. ;)
It's things like access control systems, especially on large sites, where you don't rip them out if they work, that are going to cause the most problems.
Of course the reality is, it probably won't be a problem like the Y2K bug wasn't really, and that'll be down to the (highly paid) hard work of some programmers who'll then get to spend the next x years hearing about how 'it was a load of bollocks anyway'.
"There were plenty still in use in 2000 from the 70s and 80s. Computers since then have become so much faster that their useful lifespan has only increased."
The problem may be less the computers and more the data. Even in the 90s we were mostly using computers that were immune, but we were constantly having to read (and sometimes write) file formats that were problematical.
I'm not in the banking or insurance trade, but the impression I get in these columns from those who are is that they have enough trouble migrating to new (that is, not unsupportably old) hardware without worrying about the databases being processed by said hardware.
This is a good point. Some friends were tasked with overhaul of an large system at the time which lacked many modern considerations. They ended up building an infrastructure to handle data migration piecemeal through user i/o via familiar tools while writing to a new database. Automation was used for what could be handled, but other records were flagged for review, allowing staff to individually sanitise during normal processes.
And here you are saying "the Y2K bug wasn't really", while (perhaps) knowing that there were many programmers who worked on the problem, thus perpetuating the myth. It was a real problem, solved by real people doing real work. Even after that, there were still systems that displayed the year 19100 (or 1900) for a while after the year changed.
>, it probably won't be a problem like the Y2K bug
Except the Y2K bug was only a problem for financial systems etc which dealt with a date and year.
Whatever the panic consultants said - no aircraft avionics software was calculating engine settings using a 2digit year
But how may SCADA systems are assuming that time_t always increases, and will do bad things if = measurement/(time_t - prev_time) returns negative, or NaN ?
And to back up your point, not in avionics, but today I've run into a problem because I updated our TLS certificates last week and one client has an Android tablet which is older than the CA's root cert and does not appear to easily support adding trusted roots.
"Confidently wrong" is never a good look.
In the second half of the 1990s, I was a product manager on Oracle Forms, a 4GL supporting tens of thousands of end-user seats (maybe even six figures, Marketing closely guarded the number...) Our users were in just about every non-real time field imaginable*. Oracle's Applications suite at the time was also built on top of Oracle Forms, and included things like supply chain, Sales and Marketing**, and HR, which very much "dealt with a day and a year".
Before that I worked in real-time systems, including supporting stuff like the developers of a missile test monitoring system***. And one of the interesting things is that they are often composed of multiple parts from multiple suppliers, and even if the top layer of software is calculating everything correctly, there can be a component buried deep in the stack that is not - and sometimes doing it wrong in hardware or firmware.
Compare also the GPS week rollover bug, which in 2019 caused problems for Honeywell's flight management and navigation software as well as certain atmospheric telemetry instruments, resulting in launch postponements.
* There were even some that acted as non-real time GUI configurators for a real time system such as industrial/commerical HVAC
** I recall the first time I was summoned to a meeting about the new "S&M application". Imagine my disappointment.
*** "...and then I thought, what if it's a bug in the compiler?"
But how may SCADA systems are assuming that time_t always increases, and will do bad things if = measurement/(time_t - prev_time) returns negative, or NaN ?
If that's the logic they're using, those systems will have already encountered this problem. Assuming time is always monotonic and increasing means that any clock adjustments backwards would cause the same issue.
These adjustments could be due to daylight savings or just normal adjustments from NTP, etc.
A great example of this is traditional telcos. AT&T is still in the process of retiring their #4ESS long distance telephone switches from the late 70s and 80s, replacing them with...a software version that emulates a #4ESS. There are many old systems in the PSTN, some countries like the UK are in the process of retiring them (System X). But in the US, many landline telephone providers are using the digital switches that were installed decades ago to replace electromechanical systems, with the oldest edge cases being from the late 1970s, the popular ones such as the #5ESS running UNIX. The idea these telephone companies seem to have is to decommission these old systems when they stop offering traditional POTS (why spend precious dollar$ to upgrade something that will be torn out when POTS is gone), which given the scale of the situation, does not seem feasible within 10 years. It took until 2017 for the last analog (software controlled relay) system to be decommissioned here. It is not just telcos--although they are a great example, many other companies have frightfully old systems lurking in various places, and if those companies have not already spent the money to replace those systems, what difference does 10 years really make when a system has been installed for 35 years quietly humming away doing its job? Certainly if management has been fine with such an old machine, why do they need to replace it *now* when it has been working and maybe even generating revenue for all this time? The scale of the problem is undoubtedly larger then one would like it to be.
That could be me!
I built a beaglebone based one which is armv7, which is 32 bit. It has lasted 10 years already so not inconceivable it will last another 13. Me, maybe not, so it will be someone else's problem if I break first. And I did just check out of curiosity. time_t is a 4 byte int, so rollover coming in 13.
Signed integer rollover is undefined behaviour.
It's undefined because when the standards were written, the existing compilers - and hardware - did multiple different things.
The standard could not ban any of them.
While the most common behaviours when adding one too many times is to either roll over to the largest negative number or clamp/saturate at the largest value, the fact both the programmer and compiler "know" it's impossible means that checking for it will be omitted.
So this is a C compiler on a non-Unix OS; does this apply on a 32-bit Unix box? I haven't seen one of those in a long time - does anyone have one to test it on?
There seems no reason why the year should matter to a simple counter until it rolls over. It sounds more like a bug in the calculations converting the counters to date and time. ISTR there was a report of Sun's OS crashing at some particular year far from 2038 - 1986 or 1987 I think - due to such a bug.
I had a sparc station 20 or something like that which ended up with a system date in the 1800s. The link loader that most programs depended on didn't like it. We ended up getting the time set to 1970 and everything worked fine.
The problem with the system in the article looks like a C library that tried to force unix time conventions onto hardware that had its own time idea and the edge cases weren't tested.
These y2k articles remind me that I need to go find the old file of companies that asked us to send reports about our y2k compliance. They seem to be entertaining for the younger staff. Maybe we should ask AI to write return letters for us.
I have no idea how the system got the odd time and the guess was something like memory corruption. It was less than the lowest 32 bit time_t. It had been running fine then nothing dynamically linked would load. The 64 bit time_t will do +/- a few billion years but most apps on Solaris of the day were 32 bit with a 32 bit time_t. We couldn't run the date command to set it and I ended up making a small statically linked program to set the time to 0 to fix the problem.
I would think the support library code of most pre ISO/ANSI C89 compilers would be crawling with bugs.
I can imagine the code in localtime(3) checking the value passed to it by reference (in seconds since the epoch) is greater than zero and zeroing if less than zero.
The error return for time(2) is inconveniently (time_t)(-1).
I suspect most time library code of this vintage also omits the year 2000 leap year (as I recall SGI Indigo Irix 4 did) and would be a day out after 2000.02.28.
I wouldn't have thought too much 32 bit code from the 1980s would in the offing in three decades time. I would more than happy to still be around to find out. :)
This bug report is competing for the title of worst bug report ever, no way to reproduce it with the information given.
The RSX-11 GTIM$ system call returned the number of years since 1900 as a 16-bit number. Most PDP-11s' only timekeeping was a periodic interrupt at the mains frequency (you could get optional hardware for shorter intervals) and didn't store the time or date across reboots. So, in principle, the OS didn't have a Y2K problem internally. It did, however, mostly use a 2-digit representation for years in the UI.
Some later PDP-11s had a TOY (time-of-year) clock which claimed to be leap-year accurate up to 2100 but only actually provided two BCD digits of year information.
There was an update from Mentec (who took over the PDP-11 from DEC) in 1998 which included Y2K compliance - but that would not in itself have affected the compilers. I think DEC C for the PDP-11 came relatively late. C wasn't much used inside DEC and I think those customers using C on operating systems other than Unix were mostly using products from third parties.
I suspect most time library code of this vintage also omits the year 2000 leap year (as I recall SGI Indigo Irix 4 did) and would be a day out after 2000.02.28.
I dunno how common that particular error would be? To screw things up that way, you have to simultaneously know that (in the Gregorian calendar) "century" years are not leap years, but also not know that years evenly divisible by 400 are leap years. That is to say, you have to be a little ignorant, but not completely ignorant.
If you're completely ignorant of the problem, you'll say that 2000 should have a leap day, and be correct for the wrong reasons.
I personally wrote code back in the 1980s where I took the attitude of "if I just go with every 4th year, it will be accidentally correct* until 2100, and there is absolutely no chance this will still be in use by then". I suspect I was not alone.
If I were writing today, I might well take the attitude "if I just go with every 4th year except century years, it will be accidentally correct until 2400, and there is absolutely no chance civilization will still be around by then."
*Accidentally correct is the second-best type of correct, after technically correct.
...it will be accidentally correct until 2400, and there is absolutely no chance civilization will still be around by then."
The way things are going, I might make that "If I just go with leap years never happening, it will be accidentally correct until 2028 Feb 29, and there is absolutely no chance civilization will still be around by then."
(Humans, probably. I'm not so sure about civilization. My code to handle calendar conversions over a +/- 25 quintillion year span [0], from well before the Big Bang to roughly after white dwarfs have cooled down, was probably overkill.)
[0] I realize only a fool writes his own date/time code. However, I'm an astronomer; the usual libraries handling 1900 and after, or at best -4713 to 9999, didn't suffice. Admittedly, a few million years -- which is what I had back in the days of 32-bit integers -- would have been enough.)
Not necessarily - wouldn't be the first time the "obvious" and "important" problems have been fixed in terms of future cashflows, but turns out there's some code on the periphery that uses today's date that will blow up the application or cause invalid queries to be generated...
99% sure there be big problems where the code just no longer exists. Running, compiled binary, sure, but no code.
Dev left (let go, retired, died or outsourced and thought “IGDAF”) and per HR policy, the laptop was wiped. That was 3 years ago. Code was in VB … 5 and running in a VM. But this code, calculates some super important calculation that powers other calculations that calculate stuff and …. $$$$ profit. Or it’s in some other language with only a handful or practitioners. BTW, does anyone need any Foxpro 2.6 or later versions updated?
It not what you know, or what you know that you don’t know. It’s what you don’t know that you don’t know that will bight yer on the bum.
There is a helpful list of problems, past and future, here:
https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs
I my own case it is a specialised DOS program that I know works to 2040 at least but only if the dosemu version works on the Linux OS, which is a problem as it depends on the vm86 instruction to emulate interrupts with is on 32-bit CPU mode only...
The Y2K bug was taken seriously by everyone at the time, and as a result a vast amount of resources were thrown at it by industry and government alike.
The problem is that the effort was so successful, people now consider it to have been an overreaction, or worse, a hoax! You don't have to dig too far down into the swamp of social effluent to find people saying this.
Anecdotally, I've found the younger people in the tech world really don't think it's a problem - and that worries me.
The nice bit about Y2K was we at least got to have a big NYE party. Maybe we need a UNIX Epochalypse party to focus the minds? Haven't done 3am in a while but I'm kind of tickled with the idea of a bunch of mates, plenty of beers, and massive 32 bit clock on the wall counting seconds in hexadecimal. Maybe just for fun I'll set all the lights to go off when it rolls over...
I was on duty that evening. At midnight we nodded, saluted each other, and went to work checking everything was running smoothly. Having had no alerts whatsoever, at around 1AM I went to the nearest ATM and checked my balance and latest account movements (an account in another bank than the one I was working for at the time). There was an interest credit of around the equivalent of 3000€. Resisting the urge to spend it there and then, I went back, showed the slip to my co-workers and pondered on what would happen from there on. At 8 AM, after an uneventful night on the job, I went down and checked my balance again. Without a trace of that earlier payment, it now showed the correct and, unfortunately, much smaller interest deposit...
Someone's night was indeed a lot more eventful than mine ;)
Personally I spent Y2K ensconced in an office with a line manager and a team leader babysitting our servers. I didn't mind missing out on all the celebrations of the millennium, which were in any case a year early.
Owing to the fact that I was getting overtime at six times my hourly rate.
That took any sting out of missing an excuse to go out and get sozzled.
Similar here. Being at the time young, single, and with the kind of social life that makes even walled-up hermits say "wow, what a loser", I had no problem volunteering to staff the Y2K "crisis centre" at my employer on the night of 31/12/99* in anticipation of a wave of panicked phonecalls from customers.
We sat around from 6pm onwards and enjoyed the company-provided food & drink and were untroubled by calls, except that there was a minor flap at 11pm when our colleagues in the Paris call-centre found that the database app we were using to track Y2K issues was not, itself, Y2K compliant and had quirked out on them!
At just after midnight our manager, conscious of the escalating overtime bills, sent us all home into the gray, chilly future wonderland of 1/1/00*.
(*) dates here expressed using two digits, for authenticity :)
"a vast amount of resources were thrown at it by industry and government alike."
Indeed. It was a scammer's paradise and an awful lot of people realised this at the time. Estimates of the cost of fixing everything were inflated ten-fold and if a few people were doing honest work fixing genuine problems then they could hardly expect to have been recognised for it, hidden as they were in a crowd of obvious crooks.
Human nature hasn't changed, so I expect we'll see a re-run in the middle of the next decade.
Y2K was an opportunity to get rid of any bit of old kit and replace it new shiny gear. Perfectly good hardware was dumpstered. The C-suite would sign off on just about any request with the Y2K compliance justification.
ImHO the big tech crash would have happened 2 years earlier if there had been no Y2K investment.
I feel since Y2K ended up being a bit of a damp squib, the “next” one will be under appreciated. The lesson not learned was in managing technical debt on an ongoing basis. As a friend once said “we beat our code into submission and feel best about its abilities the day we switch it off”.
"if a few people were doing honest work fixing genuine problems"
A lot of people were doing a lot of honest work looking for genuine problems, and fixing those they found.
Mind-numbing work it was, too. I only had a small taste, but it was enough to make me glad I won't be in the trenches for the next one.
To whatever extent there was snake oil being sold -- I don't recall that, but of course that doesn't mean it didn't happen -- the fact that (almost) everyone had left things so late made them, in their desperation, easier marks for the hucksters.
"The Y2K bug was taken seriously by everyone at the time, and as a result a vast amount of resources were thrown at it by industry and government alike."
It was only a current issue in the minds of management and the scammers/swindlers that prey on them.
The cold, hard reality is that I and several hundred thousand (a couple million? Dunno.) other computer people worked on "the Y2K problem" for well over 20 years, on and off. Come the morning of January 1st, 2000 damn near everything worked as intended ,,, thus causing brilliant minds to conclude that it was never a problem to begin with.
HOWever, in the 2 years leading up to 2000, I got paid an awful lot of money re-certifying stuff that I had already certified to be Y2K compliant some 10-20 years earlier. Same for the embedded guys & gals. By the time 2000 came around, most of the hard work was close to a decade in the past ... the re-certification was pure management bullshit, so they could be seen as doing something ... anything! ... useful during the beginning of the dot-bomb bubble bursting.
When I started work in 1987, one of the first things I worked on was a DEC VMS based accounting system and converting it from 2 digit years to 4 digit years on COBOL, which was a lot of work! You couldn't just change one module, every module that read from or wrote to a specific date in the database had to be changed at the same time, so there was a lot of testing to find out every last screen, report, batch job etc. that interacted with each date.
Everything I worked on after 1987 was designed from the get-go to work with 4-digit years, but older programs (the accounting suite came from the 1970s) needed a lot of work. Luckily, by the time we were working on it, drive sizes had increased and the drive prices had decreased enough that it was financially sound to switch them over. I remember some other, older hardware systems, where the storage couldn't be expanded were switched to "sliding windows", where there was a cut-off date, years before a certain value (E.g. 30) were considered to be 20nn and those after the value were considered to be 19nn years.
The attitude problems are hardly new. There were warnings about Y2K for years, but they were voices crying in the wilderness.[1]
But then everyone suddenly got religion about the problem in 1996 or so, and from then on there was feverish activity until the Big Rollover.
The thing is, I seem to recall occasional articles, even while the Y2K furor was in full swing, pointing out that there was another one of these barreling down the track, due to arrive in 2038. They were, once again, voices crying in the wilderness.
If history is any guide, expect a sudden panic come 2034 or 2035. If people learn from the past and wake up by 2030, I'll be most pleasantly surprised.
[1] The open-source RCS package was made Y2K-compliant in, I want to say 1990 or thereabouts. I remember being impressed, thinking: "Wow, they're looking really far ahead." (RCS stood for Revision Control System. It was an early VCS -- unbearably primitive by modern standards, but pretty hot stuff in its day.)
"expect a sudden panic come 2034 or 2035."
Panic in the eyes of your ignorant management doesn't represent an emergency on my part.
Note that most important thingies[0] with 2038 issues have already been fixed. Except certain hardware. For example, is your car 2038-compliant?
[0] There I go, using technical terms again ...
I remain pessimistic that 2038 will be addressed with any particular zeal beforehand. It was easy for people to understand y2k, and back then a lot of people had a more general distrust of computers. Explaining a 32 bit date rollover is eye-glazing for a lot of people, and in many companies an amorphous problem a CEO does not immediately understand is not going to get more than a token amount of money / an AI agent trusted to fix it.
Still, it will be toward the end of my working life, hopefully there are some profitable contracts available in the cleanup if not the runup. Appreciate any tips from those of you who ran the last round...
We were working on Y2K problems in 1987. Financial and personnel systems were some of the first that were affected, using 2 digit dates calculating retirement age, for example, had people retiring before they were born or mortgages/loans that would mature before they had started!
Anecdotally, I've found the younger people in the tech world really don't think it's a problem - and that worries me.
I've noticed this same laziness as the years go on...
1. There's no need to worry about how much memory is being used or cleaning up after yourself; the machine has lots to spare and the garbage collector will get around to it sometime, probably.
2. We now accept that a reboot is required every month/week/day anyway. Gotta keep those patches up to date!
3. Things just break, but it seems to be working fine now that I've rebooted. I have no idea why or interest in tracking down the root problem. The reboot erased all clues.
Just last week I had to urge a co-worker to tap the brakes... they were ready to go ahead and re-install the OS because a VM wasn't working, when the underlying issue was a Group Policy Object that was no longer applying. Turns out that's what happens when you move a computer out of the OU it was in... you know, like it warned you with that dialog box you clicked Ok on without reading.
It's like the schools have decided it's better to teach the framework du jour rather than instill a sense of curiosity, fundamental troubleshooting skills or scientific thought process.
"...application programmers not taking into account two digits."
In many cases that would be because all the date functions in the language they were working with were six digit only. If the system calls didn't have any way of returning the four digit year and all the manipulation / conversion / comparison functions were six digit only, you didn't have much choice.
Much of the affected stuff was written in languages developed in the '60s and 70s, when memory and storage were hideously expensive and in short supply, so nobody used four digits where two would do the job.
In the place I was at the time, we chose "60" as the window for Y2K work. The company didn't exist before that, so anything "before" 60 was 20something. I heard of one place that chose 30 for some reason. I now wonder how their humungous "replace it all" project went/is going and if they're getting nervous?
I know of several companies who would have had those issues ... but their hardware, OS and software have all had complete overhauls at least once per decade over the last half century. None forsee any problems in 2038.
Stuff that's out of reach (satellites, deep space stuff, etc.) just gets dealt with. As do legacy systems that can't be swapped out with modern kit for reasons.
The rest gets junked and replaced. Simples.
One of the reasons we had to frequently include "20" at the beginning of year fields shortly after 1998, is that someone patented the "year window" idea that would have let people use 99 for 1999 and 00 for 2000.. See: United States patent number 5,806,063.
Yes, people were using it in public, well before the patent was issued, but it wasn't worth the effort for most to fight that patent.
"One of the reasons we had to frequently include "20" at the beginning of year fields"
I think it was Verity Stob that humorously cited dates such as 19100 (for the year 2000) due to the habit of devs pasting the two digit century onto the front of the returned year. Not realizing that some date libraries would return year 2000 as "100".
I worked on many COBOL systems in the late 80s and early 90s, it was just the "PICTUREs" that needed adjusting (and a program to change the database value, before the new code with the new PICTUREs came online.
The problem wasn't necessarily the system calls, it was the lack of storage or the cost of the storage, when these systems were written, back in the 1970s, those extra 2 digits for years, across a whole database, would have been very expensive. As space increased and prices came down, it became viable to use 4-digit years.
30 was often used for personnel systems (you couldn't have people of working age being born after they retire), for example. Although even that was a little high, as someone born in the 1930s could still be working in the 1980s...
I wonder if (assuming that there are) alien civilizations universally evolve a binary computer system. Is is because of a universal concept of on(something present) and off (something not present) or is it due to two (eyes, ears, arms, legs, etc). I wonder if a starfish thinks in base 5 or a octopus in base 8? I guess we've had base 10 for a while and I guess there are some early (and not so early generation) analog computers out there. Even MLC and TLC Flash ultimately present a binary value. It even seems like AI algos try to distill binary concepts into something where humans mistakenly interpret a combination of knowledge/facts as "intelligence".
Or should I have just have another pint and think of what I am going to what on the telly next? Inquiring minds want to know....
Meso American calendars were based on two cycles running in parallel, one having 13 days and the other 20. They also tracked solar time and a sort-of 365-day year (360 days, plus 5 intercalary days during which it was unlucky to be born). They also had calendrical cycles on century-ish order of magnitude, and cycles on geological scales. Situating themselves in time was very important to them.
> should I have just have another pint
I think that would be best. -->
> universally evolve a binary computer system.
I started on 1-bit logic.
Binary is about the simplest (and smallest hardware) system IF you do not already have ten stubs to work with.
Dad said that G.E. worked on a trinary system, and Google says he was not kidding me, but for tubes/transistors (and simple relays) trinary costs more.
Binary is universal, because it's easiest to represent in physical hardware - on or off, present or absent etc.
However, all other systems will vary. Ten is an accident of bilateral symmetry with pentadactile appendages.
Modern flash can have multiple levels, I think the current state-of-the-art is 32 levels. Obviously this reduces reliability, as the smaller tolerances mean tiny charge leakage changes the value read back.
Signal to noise ratio. And economics.
Binary requires only one "guard band" to differentiate between its available states. Even then, the occasional error creeps in. Once the threshold level problems have been settled, binary requires the smallest bus width to represent some value. I can count to 1023 on my fingers in binary.
I think binary is used because it's the most mathematically simple. In typical usage, a different base doesn't change anything because the operations are the same; addition generates the same result regardless of base and all the other arithmetic operations are based on it. With purely mathematical computers, you can use whatever base you want, and some early computers were decimal. Changing those to binary made them easier to build without changing any results.
Some other operations make a stronger case for binary. Boolean logic is useful, and it's entirely based on binary true/false values. There is ternary analogue to boolean logic, and computers have been built using it, but there are two problems with it: most of the time, the third state is not used so it's a waste to keep storing it and there's not so much standardization about what the third state is or what effect it should have on other things. Since binary is the simplest base available, you can encode booleans in any base you choose, but for efficiency's sake, it helps when that's the system's base too. Many operations act by comparing large sets of booleans together very quickly, and binary logic makes that efficient.
The Harvard Mark I used base 10. One of the operators' tasks was to program in constants by setting decade switches, one switch for each decimal place. It had cabinets holding rows and rows of such switches, one row per constant.
You can still see it (well, some of it) in the lobby of the student union at Harvard.
No, really it is.
Back in the 1900s, most things worked just fine if the date was set incorrectly, and very often it was. That is the main reason why the world didn't end on 1st January 2000, as I predicted it wouldn't.
The world won't end in 2038 either, worst case scenario is something like Crowdstrike, which did happen, and didn't cause the world to end, though it did cause a lot of disruption.
Having said that, now, if the date is set incorrectly, most things will not work, because ssl certificates will be identified as being out of date, and it is embedded systems where this is most likely to happen.
True story:
Once walked into a customer's network that, overnight, had literally just stopped.
It was one server running a school (normal in those days) and it just wouldn't do anything. Frozen.
Hard-rebooted it. Booted into Windows. As soon as it got there, same thing. Safe mode: same thing.
Services would work and the thing would ping for a few seconds but once it got into Windows it would just all stop.
No problem, restore from backup. Yesterday's backup: same thing. Last week's backup: Same thing. Last YEAR'S backup: Same thing.
No way, because those backups were FINE and were tested and restored perfectly just recently.
Tries a Live CD... server was fine.
After much hair-pulling, I found a hint in an online forum.
Turns out that the APC UPS software was written in Java. One of its JAR files held a certificate. That certificate silently expired (and no way for a customer to renew it anyway). When it expires, rather than just error, the APC software absolutely jams the server up so hard that you literally can't do anything, not even get a ping out of it.
The fix was to modify the filesystem in another OS and remove that JAR, then the software would just stop working rather than mess everything up.
Then you could uninstall the UPS software and install a later version and it would be okay... until the certificate expired again, I suppose. I had a very annoyed customer over that one, because they were entirely down and there was nothing obvious at all, and backup restores are NOT things you can do quickly, and a few of them and STILL no success? Nightmare.
In other news, a very important MS certificate just expired and people have been scrambling to fix it, but fortunately that doesn't bring your machine to a grinding halt and Windows Update mostly takes care of it so long as you update regularly enough.
Some applications rely on having an accurate time.
However at the TCP/IP level it's irrelevant.
HTTP doesn't care what time it is either, though setting you clock far in the future would result in your browser crying about expired certificates for HTTPS connections. Also the browser probably wouldn't cache anything due to Expires dates in the "past".
I would have expected it to reset to 1970, as that is what computers have always done, when the battery backed RTC fails, or on older PCs, where there was no battery backed RTC and you had to enter the date and time every time you started the computer.
I would expect unpatched Y2K programs to loop back to 1901, but UNIX time was always anchored in 1970.
The Epoch bug is going to be huge - much worse than Y2K. This is because -
* It will affect anything using 32bit time - which is a LOT of legacy systems, including many embedded systems, popular libraries, etc, that people may not even be aware of
* It is much more than just a display/parsing issue; while Y2K did have storage isues on older systems, a lot of it was just converting the year for human reading/writing
* Since we worked so hard to prevent problems with Y2K, nontech people (managers) will expect this to be a nothing as well
* 2038 is not a nice round number and so nontech people (managers) will not expect anything to go wrong
I will be retiring around 2038, and so before I do, I will make stonking amounts fixing Epoch bugs everywhere on egregious consultancy fees...
Back in 2000, we worked hard and despite the paranoia of management, only one thing failed after Jan 1st -- the fancy Windows Y2K countdown screensaver that was commissioned by management from outside contractors and mandated to be force-installed on everyones desktop. Apparently, nobody had tested what it would do after Jan 1st. It crashed.
Any system that's been built or upgraded in the last decade or three now has a 64 bit time counter. Its a simple change & its easy to implement. There may well be some really old boards running 32 bit time but the chances are that they won't be using a clock/calendar except for show and they're almost certainly not going to have a file system.
Speaking from experience the glitch really isn't that noticeable anyway. If you're really worried about it set the time forward and see what happens.
>after Jan 1st. It crashed.
I presume it wasn't tested....
"Any system that's been built or upgraded in the last decade or three now has a 64 bit time counter."
Yes, but that's not the main issue. Any code that was compiled as 32bit might be running on a 64bit system, but still using a 32bit int for the timestamp value. Even code compiled as 64bit might explicitly declare a variable as 32bit and then use it for the timestamp. Without someone properly looking at the code, you wouldn't necessarily know.
And the real problem (as with Y2K) is database fields. If you've got a database with a 32bit timestamp field then again, running on 64bit hardware is not much help. Someone needs to have a proper look at the database structure and the code that accesses it.
Hi All - Just some extra information for you - the 11/73 in question is running RT-11 and TSX when multi-user, as shown here...
The C Compiler is the Decus C compiler - but the point in not that this particular compiler is buggy (unless you're in Canada with PDP-11 systems in your Nuclear power stations until 2050) - but to illustrate that so many systems are untested so far into the future - and may not be supported or updated - and "just work" - until they don't...
The 2037 problem we found at The National Museum of Computing www.tnmoc.org causes a stack overflow and program crash on any call to time() after the end of 2036 - and this is certainly unexpected and one good example of what we might experience.
Doubters might want to take a look at NASA losing a spacecraft in 2013 due to a 32 bit signed int rolling over: https://llis.nasa.gov/lesson/10701 - most of us do not have their resources
Robin
The Year 2038 issue gets shrugged off as “solved on 64-bit,” but that is only true for consumer operating systems. A huge amount of legacy and embedded infrastructure still runs 32-bit code compiled decades ago in finance, manufacturing, medical, and industrial control systems. Many of those environments cannot simply recompile for 64-bit without breaking interfaces, certifications, or binary data formats tied to time_t.
There has been an effort to standardise remediation through the Epoch 2038 Standard from the Legacy Systems Foundation. It provides a structured framework for identifying, testing, and remediating time-sensitive systems across C, POSIX, and database layers. They have also open-sourced reference tools for scanning and validating affected systems:
https://legacysystemsfoundation.org/epoch-safe
https://github.com/legacysystemsfoundation/resources
For teams that need to plan actual migration or compliance work, there is a paid Epoch Safe Handbook on Amazon that expands on the audit methodology and system design patterns: https://www.amazon.co.uk/dp/B0FNRDMW1C
It is one of the first serious attempts to treat 2038 as a lifecycle and compliance problem rather than a compiler flag, which is probably the right mindset given how much critical software still assumes time is a 32-bit signed integer.
Newbie and single-poster mikedodduk writes:
"It is one of the first serious attempts to turn 2038 into a profit center, preying on the backs of ignorant management"
FTFY
"a paid Epoch Safe Handbook on Amazon"
You don't say ... Only one review. A glowing one. By some guy named "Mike". Colo(u)r me surprised ...
