back to article Don't turn it off and on again: Expired Cisco cert cripples vEdge SD-WAN kit

An expired security certificate is threatening to wreak havoc with Cisco customers' wide-area networks. For a change, turning the equipment off and back on again will only make things worse. In a bulletin published this week, Cisco warned that customers using vEdge SD-WAN appliances could experience complete loss of service if …

  1. Snake Silver badge

    Paying attention

    Nice of them to notice AFTER the horse has bolted from the stable. Even when they're both the horse AND the stable.

    1. Danny 14

      Re: Paying attention

      its a great way of killing second hand kit. No support contract? No new ssl cert for you.

  2. claimed Bronze badge

    If you’re a device or software producer, please start rejecting certificates 3 months before expiry, then include a bypass which will get you to ACTUAL expiry with a config setting (which can’t be put in early, so disable this on startup or something).

    That should give you and your customers some recourse in these events. Yes, it means writing a couple of extra lines of code, and there might be better ways, but this way is better than nothing!

  3. Anonymous Coward
    Anonymous Coward

    Cisco can go and absolutely super-fuck itself after the day we've had un-fucking this absolute dumpster fire

    1. Ryan D

      I am sorry to hear that I’m not the only one trying to deal with this flaming turd. Your sentiment is shared with my team. Cisco support is basically nonexistent at this point.

      This is a class 10 shitshow.

  4. VoiceOfTruth Silver badge

    Nobody ever got fired

    For buying Cisco. But they should have been.

  5. Alistair
    Windows

    I think we can tag this to the

    Layoff costs from last quarter. It seems they let one body too many go from the relevant department...... /boardroom discussion

    Anyone around these parts keeping track of the number of IT related layoffs in the last two years? I'm thinking we should start calling it the 'great thinning'.

    1. VoiceOfTruth Silver badge

      Re: I think we can tag this to the

      If you're old enough to remember the 1980s in the UK, it was practically daily news about which engineering or manufacturing firms were shutting down. New jobs created were shopping centres and supermarkets. Yep, skilled jobs out shelf-stackers in.

  6. Norman Nescio Silver badge

    Workaround?

    I'm, thankfully, not familiar with this kit. Is a temporary workaround setting the clock back so the device treats the certificate as still valid, which then allows you to replace the said certificate?

    Of course, Cisco might have implemented some measures to make doing this difficult, or even impossible (e.g. certificate expiry blows a physical security fuse), but that's the kind of behaviour for sites that take security expensively seriously, backed up by people with guns.

  7. Bebu Silver badge
    Big Brother

    Feeding two birds with one scone?

    More like screwing two pooches with one dick.

    Definitely on the list I should think.

    The chap let go for having a "blind spot" about stoning a pair of birds was probably the employee who ensured the certifcates were renewed.

    Pretty clear that AI hasn't a lot of competition in the 'I' part.

  8. Frozit

    Certificates are... hard

    Sadly, the certificate mechanism almost guarantees failure. You have this hard check, that one second before it works fine with no notification, one second after, it is borked.

    You can code around it by adding in warnings and grace periods, but you know as well as I do that they will get ignored.

    One solution is what Google does, have several massive wildcard certs that expire at short intervals. This forces them to keep the certificate process active, as opposed to once every 3 years... However, creating that cert is easy if you are certificate authority. Probably quite expensive if you have to pay for one.

    We stood up a private certificate authority and forced everyone to use it in the development and test area. That did amazing things for getting developers to understand certificates. But I eventually had to make a "get out of jail free" cert that I based on the Google one. Explaining certs to the people hired as testers was sometimes rather difficult.... However, it still has the benefit of everyone knowing how to install a cert and what things look like when you have a cert failure.

    1. tip pc Silver badge

      Re: Certificates are... hard

      if your writing the software to check the cert then you can decide what state you want to reject at.

      In this case the cert has failed due to the date expiring, you could put a warning and allow the cert to still pass x (i recommend 3) months after expiry so long as its not been revoked.

      Revocation is the main thing for certs, dates just guard against bruteforce attacks as the cert should expire well before the feasible date it could be bruteforced.

      a recently expired strong cert is not a super huge risk.

      A revoked cert is a risk

      An expired cert that that should have been revoked but isn't as the makers knew it was expired would also be a risk if the checking software permitted its use beyond the expiry date.

  9. This post has been deleted by its author

  10. CBW140

    Similar issue last year

    Last year their update server TLS certificate rotation caused a bunch of CBW access points to require manual intervention, or never receive another firmware update. Not the end of the world, but much more difficult than just hitting the update button:

    https://community.cisco.com/t5/wireless-small-business/cbw-140ac-cannot-update-to-new-frimware/td-p/4580608

  11. _Elvi_
    Coat

    TITSUP

    what..

    we cant use TITSUP any more?

  12. Norman Nescio Silver badge

    Backronym

    Total Inability To SD-WAN Under Power-cycle.

    (My backronym skills are flagging.)

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like