back to article FAA grounds all US departures after NOTAM goes down

The US Federal Aviation Administration ordered airlines to "pause all domestic departures" this morning as it tries to deal with an outage in a critical computer system. The agency said in a tweet that its Notice to Air Mission Systems (NOTAM) system was offline, affecting operations throughout US airspace.  The FAA is …

  1. Mishak Silver badge

    "but which aren't known about enough in advance to publicize by other means"

    They are also used to notify events that are well in the future, such as closing airspace when a rocket launch is scheduled.

    1. Yet Another Anonymous coward Silver badge

      Re: "but which aren't known about enough in advance to publicize by other means"

      And were known as Notice To Air Men for 70years until somebody noticed it was sexist.

      There was then a long and involved process to come up with a new pronoun freindly version that wouldn't involve changing lots of software and paper forms

      Plus "Air Missions" makes you think of Top Gun while flying a 5am commuter flight to Minneapolis

      1. imanidiot Silver badge

        Re: "but which aren't known about enough in advance to publicize by other means"

        And the entire rename is stupid because I don't see why airmen is sexist to begin with. Female aviators are airmen too. That term doesn't have to be gendered in the first place.

        1. Yet Another Anonymous coward Silver badge

          Re: "but which aren't known about enough in advance to publicize by other means"

          After the slightly silly renaming of fishermen to fishers (they were always trawlermen in Yorkshire so are now presumably trawlers)

          They could rename anyone who flies a plane as "airs" or perhaps "airers". Or since there are no longer any flight engineers or navigators or radio operators - couldn't it just have been "Notice To Pilots" - again except that it would have involved changing a lot of forms

          1. Uncle Slacky Silver badge
            Headmaster

            Re: "but which aren't known about enough in advance to publicize by other means"

            > They could rename anyone who flies a plane as "airs" or perhaps "airers"

            "Aeronauts", shirley?

            1. seven of five
              Coat

              Re: "but which aren't known about enough in advance to publicize by other means"

              > "Aeronauts", shirley?

              and Aeronautresses? Plus the inevitable Aeronautteurums.

          2. Doctor Evil
            Coat

            Re: "but which aren't known about enough in advance to publicize by other means"

            My wife came up with that "Notice to Pilots" independently over coffee this morning -- except she suggested that they be called "PIlot NOTification" (or PINOT), with coloured suffixes for the gravity of the situation being notified about: you know, PINOT noir for life-and-death type stuff, PINOT blanc for more nice-to-know information, ...

            I mulled it over while having my toast.

            Cheers everyone!

            (I'll get my own coat, thanks. It's the leather bomber with the aviators in the pocket.)

            1. Tim99 Silver badge

              Re: "but which aren't known about enough in advance to publicize by other means"

              PINOT Gris for older and more experienced flyers?

              My late father said that when he was WW2 RAF aircrew: "Rules are for the obedience of fools, and to advise the wise".

            2. CrazyOldCatMan Silver badge

              Re: "but which aren't known about enough in advance to publicize by other means"

              It's the leather bomber with the aviators in the pocket

              Must be big pockets to fit a whole aviator in..

        2. Anonymous Coward
          Anonymous Coward

          Re: "but which aren't known about enough in advance to publicize by other means"

          And the entire rename is stupid because I don't see why airmen is sexist to begin with.

          Never, ever underestimate the willingness of the professionally offended to take issue with anything which even hints at gender issues.

          1. DoctorPaul

            Re: "but which aren't known about enough in advance to publicize by other means"

            Upvote for "the professionally offended" - a phrase that perfectly sums up my feelings, especially as they are usually offended on someone else's behalf.

        3. Bitbeisser
          Devil

          Re: "but which aren't known about enough in advance to publicize by other means"

          Yeah, that whole gender/pronoun nonsense is a major PITA for all of mankind...

      2. Dave559 Silver badge

        Re: "but which aren't known about enough in advance to publicize by other means"

        I agree that "Air Missions" sounds just a bit over the top: with a bit more lateral thinking couldn't they have come up with something like "Notice To Aeroplane Airplane Movements", maybe?

        1. yoganmahew

          Re: "but which aren't known about enough in advance to publicize by other means"

          There are separate Airplane Movement systems that perform different functions (tracking the airplane). FLIFO - Flight Following.

      3. Anonymous Coward
        Anonymous Coward

        Re: "but which aren't known about enough in advance to publicize by other means"

        Have an upvote, but as occasional SUF (Self-Unloading Freight) on a 5AM commuter flight to Minneapolis, believe me, there's no way to make it feel like Top Gun!

      4. ricardian

        Re: "but which aren't known about enough in advance to publicize by other means"

        The RAF recently changed the names of some ranks:

        Leading Aircraftsman (LAC), Senior Aircraftsman (SAC) & Junior Technician (JT) became Air Specialist (class 2), Air Specialist (class 1) and Air Specialist (class 1) Technician.

        Source https://www.raf.mod.uk/our-organisation/raf-ranks.

      5. SotarrTheWizard
        Holmes

        Re: "but which aren't known about enough in advance to publicize by other means"

        According to ICAO, it's still "Notices to Airmen". The PC version is purely US. . . .

      6. Anonymous Coward
        Anonymous Coward

        Re: "but which aren't known about enough in advance to publicize by other means"

        5am - would that be a nocturnal air mission?

  2. Vulch
    Black Helicopters

    So...

    A "Who, me?" in about a decade...

    1. Anonymous Coward
      Anonymous Coward

      Re: So...

      I'm starting to think there should be a "Who, them?" feature for people to detail the failures caused by Bean Counters/ Inept Manglement.

      1. Boris the Cockroach Silver badge
        FAIL

        Re: So...

        El-reg would dso full of these, there would not be enough space to publish anything else...

        Now... did I tell you about the beancounters and the new purchase system

    2. bombastic bob Silver badge
      Devil

      Re: So...

      Some possible phrases within that "Who, Me?" article

      * What version of Micros~ Access was this written for? I need to re-install it

      * Windows 10 was updating? AGAIN???

      * What do you mean, "no daily backup"?

      * 'Please insert disk in drive A: and press OK'

      * That is most certainly a BSOD

      * 'mailbox full'

      * Can anyone find me a working IDE drive?

      * CPU fan was clogged

      1. First Light

        Re: So...

        * The cleaner pulled a plug

  3. Dr. Ellen
    FAIL

    Pot? Kettle here.

    This will make it far more difficult for the FAA to scold or fine Southwest.

    1. DS999 Silver badge

      Re: Pot? Kettle here.

      Couple hours of halt versus days of chaos and people sleeping in airports for several days in a row. Not quite the same thing.

      The NOTAM system was also nice enough to go down after the holidays were past, and on one of the lightest flying days of the week. And in the early morning, so few if any people will end up stranded overnight for even one night.

      I am curious if this was a hardware failure where redundancy failed to kick in or a botched software upgrade. Overnight between Tuesday and Wednesday is exactly the time one would pick for such an upgrade, as it would incur the minimum potential disruption.

      1. ecofeco Silver badge

        Re: Pot? Kettle here.

        The delays and disruptions are expected to last for days. For ALL airlines.

        You were saying?

  4. Anonymous Coward
    Anonymous Coward

    Just after Patch Tuesday? Hmmm.

    No, no, surely that's purely a coincidence.

    Let's not mention the fact that Friday is also the 13th. I mean, I don't want to worry anyone..

    /s

    1. Yet Another Anonymous coward Silver badge

      Re: Just after Patch Tuesday? Hmmm.

      You think the FAA have updated to windows?

      I'm betting a 2000s era IBM mainframe emulating a 1980s IBM mainframe running an IBM mainframe OS from the 60s with an app written in System360 Assembly

      1. yoganmahew

        Re: Just after Patch Tuesday? Hmmm.

        They're probably running at least z15s. Program design of 1960s assembler bears a sharp resemblance to current vogue of microservices architecture and is possible to build error free and understandable to future generations - you don't have to endlessly refactor it to understand it. The documentation has not kept pace, but I currently work on java systems that are built with no documentation (not even comments in the code) other than a story.

        1. CrazyOldCatMan Silver badge

          Re: Just after Patch Tuesday? Hmmm.

          My first job was a TPF assembly programmer (on IBM 3090J iron). Our (inherited) codebase had stuff dating back to the late 1960s..

          And some of it generated self-modifying code. We had an interesting time cleaning *that* up! Some of those early segments were obviously programmed by people obsessed with keeping the code as small as possible, even at the expense of maintainability..

          1. Don Casey

            Re: Just after Patch Tuesday? Hmmm.

            My first job as a mainframe programmer (1971) was for a 360-40 that had 64K (yes, K, not M, not G) of memory. This was before 'virtual memory' existed.

            That 64K included memory for I/O buffers and the operating system. YOU HAD TO KEEP CODE SMALL!

      2. Doctor Evil

        Re: Just after Patch Tuesday? Hmmm.

        "I'm betting a 2000s era IBM mainframe emulating a 1980s IBM mainframe running an IBM mainframe OS from the 60s with an app written in System360 Assembly"

        And if you've ever actually read a NOTAM, you would believe this to be true. They look as though they're intended to be distributed via teletype (as they probably once were?), character-limited to minimize the cost of transmission.

        1. Doctor Evil

          Re: Just after Patch Tuesday? Hmmm.

          For the curious ...

          https://www.faa.gov/sites/faa.gov/files/about/initiatives/notam/what_is_a_notam/Pilots_NOTAM_primer_for_2021.pdf

          Example (from the reference):

          IAP LOS ANGELES INTL, Los Angeles, CA. RNAV (GPS) Y RWY 24L, AMDT 5... LPV DA 628/ HAT 505 ALL CATS, VISIBILITY ALL CATS RVR 6000. LNAV/VNAV DA 632/ HAT 509 ALL CATS. TEMPORARY CRANE 342 MSL 5513FT EAST OF RWY 24L (2016-AWP-6554-OE

          1. Tempest8008

            Re: Just after Patch Tuesday? Hmmm.

            I work in the industry and I can read that. They're kept short and to the point because pilots, at a literal glance, can get all the information they need while actively piloting the vehicle. More interesting are the ones where the airfield has significant snow clearing operations. Cleared width, height of snowbanks, which side of the runway has any windrows, the RWYCC (Runway Condition Code) telling the pilot how slippery the runway is likely to be when braking....it's all in there!

            1. Anonymous Coward
              Anonymous Coward

              Re: Just after Patch Tuesday? Hmmm.

              They appear to have an unhealthy cat fixation, though :)

              Joking aside, I am actually very happy with the efficiency and, in a way, the "backward" appearance for a simple reason: it works. And it works very well indeed.

              What evidently did NOT work was resilience and resilience testing - this system or a clone of it should have been kicked offline to see how well it recovered. On the plus side, again, is that backups have backups there, and people continue to be trained in the manual fallback processes that it takes to get everyone safely on the ground.

              Yes, it caused a massive disruption, but note that the airspace closed safely - let's be grateful for that.

        2. yoganmahew

          Re: Just after Patch Tuesday? Hmmm.

          They still are (can be) distributed by Teletype...

  5. John Doe 12

    F.A.A.

    Feck all aircraft - in the sky :-D

  6. rcxb Silver badge

    'no evidence of a cyberattack at this point'

    Thank goodness it's just the usual level of incompetence.

    1. Yet Another Anonymous coward Silver badge

      Re: 'no evidence of a cyberattack at this point'

      This is not just ordinary incompetence, it's special Federal Agency incompetence

  7. ecofeco Silver badge

    Wait, I thought the SWA guy was promoted

    Did he actually get hired by the FAA?

  8. martinusher Silver badge

    Old clunky system goes down

    The NOTAM system does the same job as a bulletin board. An older bulletin board that relays messages by Telex /(because that's what the messages look like).

    It shows just out of touch the FAA is. The obvious answer to not being able to issue a NOTAM isn't to ground all domestic flights but to stop creating the damn things in the first place. Very little would have happened overnight that would affect airline traffic -- the notices are primarily about prohibited areas, exclusions zones around Presidential movements and major spectator events (they're the only things that crop up at short notice that people don't know about from other sources) so delaying whatever would cause the NOTAM for a few hours or even -- SHOCK! --- taking a risk not grounding all traffic with in a 30 mile radius might work just as well.

    But if you've got the Power then its a pity not to use it, I suppose.

    1. An_Old_Dog Silver badge

      NOTAMS Alternatives

      the notices are primarily about prohibited areas ... That seems like an important thing to pilots, if only to avoid having the FAA yank their licenses for violating said prohibited areas. And there's nothing wrong with printed Telexes, or electronic bulletin-board systems from the 1980s, if they get the job done.

      I notice the FAA has a web page where pilots can search for NOTAMs relevant to a specific flight (you enter the origination and destination airports, or a user-defined radius from an airport, etc. [https://pilotweb.nas.faa.gov/PilotWeb/].

      Sure, it seems like grounding those flights was a misuse of The Powah, but I wouldn't have bet my life on the chance all NOTAMs my pilot missed (due to system failure) would have been "irrelevant."

      1. Yet Another Anonymous coward Silver badge

        Re: NOTAMS Alternatives

        So air-persons subscribe to messages about specific routes and destinations?

        The messages are short and consist of abbreviations that nobody outside the club understand.

        I'm thinking of an entirely stable and reliable commercial platform that provides this sort of capability at scale.

        And emoticons on NOTAMs would be fun

        1. SotarrTheWizard
          Holmes

          Re: NOTAMS Alternatives

          No. You look up airports on and along your route.

    2. dl1jph

      Re: Old clunky system goes down

      You've conveniently forgotten some of the more important categories - temporary restrictions on airfields, for example. Most of that is short-term and known only a few hours to days in advance, yet it can very significantly alter the flight plan for the pilots - a mayday fuel isn't fun at the best of times but getting into one because you didn't know some low-hanging clouds and a precision approach system down for maintennance would prevent you from using the airports you expected to be safe to use is just stupid. Thanks to the NOTAM system providing this kind of info in one place for every bit of airspace around the world, it's also trivially easy to avoid.

      1. Don Casey

        Re: Old clunky system goes down

        For the curious; see here: https://www.faa.gov/air_traffic/publications/domesticnotices/

    3. Tempest8008

      Re: Old clunky system goes down

      This is incorrect. Weather events and runway conditions are also communicated via NOTAM. If I'm flying into Toronto and they had a storm go through an hour before I'm set to land there, a Runway Surface Condition inspection is conducted and a NOTAM is issued that tells the pilots the runway conditions. Icy, snow covered, swept, brushed and sanded etc etc. It's important information for the pilots to have and they CAN'T have that information before they take off. It's not current or accurate.

  9. KittenHuffer Silver badge
    Angel

    I think that ....

    .... little Bobby Tables is flying off on holiday somewhere!

    -----------> Little Bobby is such a little angel!

  10. anonymous boring coward Silver badge

    Single Point of Failure

    SPOF.. Ess-Poof

    I'm a not surprised, but I am appalled by the state of things.

  11. Uncle Ron

    Huh?

    Cloud? Mainframe? X86? AIX? Windows? Oracle? Unix or Linux? DB2? I'd sure like to know these details. Somebody knows.

    1. Anonymous Coward
      Anonymous Coward

      Re: Huh?..and its a Buzzword bingo "Modernization"

      https://www.faa.gov/about/initiatives/notam

      https://www.databricks.com/session_na20/automating-federal-aviation-administrations-faa-system-wide-information-management-swim-data-ingestion-and-analysis

      Its The Cloud.... Its Java Messaging... What could possibly go wrong.

      Bring back TSO/E on a good old reliable IBM 370 I say. It worked.

      1. bombastic bob Silver badge
        Devil

        Re: Huh?..and its a Buzzword bingo "Modernization"

        The early rumor was "a corrupted database file".

        Made me think it might have been a '.mdb' file, heh heh heh

        But as it's gummint, probably an 'Oracle solution', The articles you mention seem to be about a proposed system and/or data analysis, unless I missed something.

        (The longer link did mention Oracle once, also Sybase)

  12. Anonymous Coward
    Anonymous Coward

    Our recent failure wasn't DNS...

    (Anon due to being work-related...)

    We recently had a major system semi-outage - three-quarters of users' computers couldn't access a particular system which is very critical to day-to-day business. But sometimes, rebooting and trying again would work. Sometimes. Apparently it depended on which domain machine the user's computer connected to at bootup. IT eventually figured out what it was. Not DNS for a change...

    It was WINS!

  13. bernmeister
    Thumb Up

    Simple system

    NOTAM was a relatively simple system using Baudot code on RTTY. I wonder what they have done to it to make it crash? I was working on a system in South America intended to get NOTAM's onto the satellite REDDIG system designed to connect all South American airports. I freaked out when an airport NOTAM system crashed so I reverted it back to original configuration. The system designers stepped in and made it work. The loss of service was so short nobodty noticed.

  14. Anonymous Coward
    Anonymous Coward

    Really...

    The truth may escape, eventually, ransomware and shite security the FAA are known for most likely.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like