back to article South Australia blacked out by bad bespoke software, not wind farms

The Australian Energy Market Operator (AEMO) says wind-powered electricity generation's “intermittency” had nothing to do with the blackouts following South Australia's catastrophic storms in late September. In its ongoing investigation of what caused the “system black” event on 28 September, the AEMO says the storm caused a …

  1. Anonymous Coward
    Anonymous Coward

    Actually…

    … big chunks of the network were knocked out by the transmission pylons being ripped out of the ground.

    Would Sir care for a wireless alternative to get all those MWh of energy to far-flung corners of the state? Thought not.

    1. MrDamage

      Re: Actually…

      Shhh, don't tell the LNP* that. How can they continue to claim coal is good for humanity if they can't spread the FUD over the cause of the failure?

      *Lying Nincompoop Party for any non-aussies out there.

  2. frank ly

    "AEMO is also less-than-thrilled about how much it didn't know regarding the ride-through settings – because vendors tried to keep the information to themselves: “AEMO will consider in the longer term the most appropriate level of disclosure and verification for settings embedded in proprietary software control systems”. "

    Disclosure is easy, you ask the vendor to tell you. However, you need the skill and experience to ask the vendor the right questions and to let them know what level of detail you want.

    Verification is difficult because you need to set up system level tests and that takes real skill and experience and quite a bit of time and money.

    Did AEMO go through all this before system acceptance?

  3. melts

    so the wind shut down due to voltage fluctuations, you claim its just a software issue?

    those I know with knowledge in the industry said wind in our currently deployed guise is the problem as it doesn't have inertia to stabilise a system, when the frequency starts to shift they need to disconnect.

    This is what you're describing, basically, but claim its just a software issue. However the inertia problem is a physical problem manifested by how the AC is created from inversion, as the wind tower doesn't spin at 50Hz, you need to create your 50Hz from electronics, and without large banks of rapid storage and massively overspecced inversion electronics a shift of just a small percentage will fry the otherwise suitable cheaper electronics they have used.

    If you haven't gathered, a dip in frequency is due to large loading lagging the network, and so needs a large injection of extra power to stabilise the network. traditional power plants have a big rotating mass that does that instantaneously - the masses inertia will attempt to maintain the frequency.

    The same electronics also means you can't start a grid on wind power, I'm sure the AEMO will find a political way of explaining why the wind power that was still spinning didn't allow for a partial grid restart once the loads had been shed.

    I'm not against renewables. No I'm against them when implemented poorly. as is this case, cheaply without the ability to synchronise and stabilise the grid. if they are going to be used like that then you need to keep the rest of the parts that can do that work. That or you need to make the wind generators provide synthetic inertia so the grid doesn't collapse when its in a bit of a pickle. Maybe battery backup will come to the rescue in that regard.

    1. mark 177

      Voltage not Frequency

      The article said the shut downs were due to voltage fluctuations, not frequency fluctuations. So how is your point relevant?

      1. david 12

        Re: Voltage not Frequency

        The report says:

        "The sudden and large deficit of supply caused the system frequency to collapse more quickly than the Under-Frequency Load Shedding (UFLS) scheme was able to act, resulting in the [total black out]"

        also:

        "This observed reduction in network voltages is consistent with a loss of synchronism between the SA

        power system and the remainder of the NEM." (That is, 'seems to have been caused by')

  4. Tom7

    Low voltage ride-through is not something you can just arbitrarily reconfigure to happen as often as you want; it usually involves dumping a significant proportion of the turbine's output power into a resistor - and they have a limited capacity to get hot before they melt.

    Wind generators are generally unhelpful in this regard. Because of the way their inverters work, they need the grid to be operating at rated voltage to export power. Any voltage dip is amplified by wind generators as their contribution to the grid also dips.

  5. Adam 1

    well called Richard

    As you predicted, a safety mechanism caused by grid issues caused these generators to perform an emergency safety shutdown. Did exactly what it should have.

    Why the grid operators and generators hadn't specifically consulted each other on what those thresholds should be is very much a live question. The fact it took them so long to acknowledge the cause is also regrettable as it allows the opportunist pollies to come out. I wouldn't hold my breath for an apology from them however.

  6. Griffo

    So in other words.. this is what is often called "A learning event". It wasn't the fault of any one trigger, but put together the loss of multiple HV distribution networks lead to an unexpected and unintended outcome.

    Just like most IT organisations took years to learn how to gracefully handle single network service outages, it seems the green power network also needs to learn through a little trial and error how to continue to provide service in the face of component failure.

    But still there's no justification to blame "green power" over coal. In reality we've just got accustomed to a network that is stable due to 100 years of trail and error R&D.

  7. Grumpy Rob

    Gosh, I like simple answers to complex questions!

    There are a few contributors who seem to think network stability is really simple, but I can assure you it's not. Once you have a succession of big disturbances on the transmission network you are in uncharted territory. It's all very well saying that big rotating machines (i.e. coal-fired turbines) have lots of capacity to "instantaneously" provide extra power, but that's conveniently ignoring the problems involved in trying to keep widely separated generators in sync - there are transmission delays, reaction times of the turbine governors, etc, etc Even if the SA generation had been 100% coal-fired there's no guarantee that it would have stayed up. So it's just opportunistic to blame wind generation.

    I certainly agree with the consensus that the introduction of lots of green power could have been better managed (hindsight *is* 20-20 vision), and griffo is right that this was a "learning event". But I get *really* annoyed by the pollies and pundits mindlessly (or is it duplicitously?) demanding cheap AND 100% reliable electricty - it's an engineering trade-off, you can't have both simultaneously.

    And on the subject of the cost of electricity - it's a few years since I looked at the annual reports of the NSW electricity distributors, and they were then 100% Government owned. But from memory the annual reports showed that the distributors paid a total annual "dividend" to the Government of around a billion dollars, plus another half a billion odd in debt repayments. But "dividend" is just accountant bullshit for tax, since they are (well, were) publically owned assets. So a lot of the blame for high electricity prices lies at the door of our pollies, rather than "network gold plating" or green energy. Don't believe me? Well get off your bum and look at the publically available annual reports.

    1. david 12

      Re: Gosh, I like simple answers to complex questions!

      >But I get *really* annoyed by the pollies and pundits mindlessly (or is it duplicitously?) demanding cheap AND 100% reliable electricty

      I was repeatedly amazed than nobody thought to draw or reject a parallel between NSW (goldplating) and VIC (bushfires due to insufficient goldplating). One state spent "too much" on network infrastructure, and didn't have network failures. The other spent "too little" on network infrastructure, and did have network failure.

  8. Diogenes

    First, the towers collapsed after the wind farms had failed to perform as required, which involves shutting down without short-circuiting the entire network.

    Secondly, even if the tower collapse had occurred prior to the wind farm failures, the effect would have been localised – Adelaide and almost all the state outside the north west would have operated normally.

    The problem has been termed a “software glitch” which prevented the safe closure when sequential problems arise. Five of the ten unreliable wind farms have now given adequate assurances that this has been rectified. It is however more than a glitch, which is why one manufacturer, Suzlon, is resisting. All generators suffer less damage if they can close immediately on facing a shock. The faulty wind generators were designed to allow only three such events to take place before abruptly closing down.

    Only a month previously, seeking to get special payments for his firms’ coal plant, AGL CEO Andrew Vesey had said, “the rules governing the National Electricity Market need to be overhauled to deal with the unreliable nature of renewable energy sources to ensure a stable supply of power to businesses and households”. He re-iterated such views in the aftermath of the most recent South Australian crisis.

  9. david 12

    "political assertions ... were false"

    >However, AEMO's updated preliminary report (PDF)

    >makes it clear that the political assertions made about

    >wind energy were false.

    However, the political assertion that I actually heard, from an actual senior federal government member, early on the first morning after, was that SA had not spent enough money on system stability, because they had spent the money on wind energy.

    I don't see anywhere in the report where it indicated that enough care was taken about system stability, and I don't see anywhere in the report where it indicates that wind energy made more money available for spending on system stability.

    Any of those things could be true. Or it could be I've been trolled by a cheap political assertion in a Register article.

  10. Mark Pawelek

    I blame everyone : grid operator and wind farm operators.

    Not a good description of the real problems. It's well known wind turbines do not work well in stormy weather. They should've had more gas turbines fired up, ready to switch in as the inevitable wind turbine shutdowns happened. As the storm approached, South Australian electricity sources were (approximately): 18% gas turbines, 47% wind turbines, 35% imports from Victoria. They did not have nearly enough gas connected to the grid and were just not prepared in any way. As if they just expected to sail through this. A second mistake they made was running the Victoria interconnector nearly flat out. With inevitable wind turbine failures, the Victoria interconnector was the only source they had that could be quickly switched in to cover for lost generation. South Australia has no hydroelectricity to speak of, which is a source that might normally be expected to fulfill that role. If the interconnector was already nearly maxed out, as it was, no reserve capacity could be quickly switched in. So: zero grid resilience.

    Of course the loony hippies don't want resilience. They'd like you all to be off-grid.

    Incompetence then. By everyone. The grid operator, wind farm operators. Yes, those who supplied the wind farm software should've made it clear their software was made to protect windfarms from catastrophic grid failure. But then, the wind farm operators should've known that. They must go back to school to study Energy 101 before being allowed out to play again.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2020