back to article Infosec experts divided on AI's potential to assist red teams

Generative AI is being enthusiastically adopted in almost every field, but infosec experts are divided on whether it is truly helpful for red team raiders who test enterprise systems. "Red teaming" sees infosec pros simulate attacks to identify vulnerabilities. It's a commonly used tactic that has been adapted to test the …

  1. Pascal Monett Silver badge
    Windows

    "We're going to have to use more and more of it"

    Well great.

    Here we have this utterly useless tool and now we find that we're just going to have to use it, instead of ignoring it entirely.

    Since when has Humanity abandoned the idea that a tool should be efficient and its use justifiable ?

    We are all just running downhill without a care in the world for what is waiting at the bottom.

    Take a step back, breath, and start thinking about where you're going and what you're doing.

  2. This post has been deleted by its author

    1. sten2012

      Re: Additional Tool - Not A Replacement

      Hard agree here.

      Huge delta between dumping all client data into an online API while having automated agents exploiting client systems and that critical headline keyword "assist".

      I think you'd be hard pressed to find a pentester or red teamer who hasn't been "assisted" by AI in some capacity at some point by now. Just with mostly appropriate care.

      How often are red teams in court anyway? Christ you've screwed up if you're in court, and AI probably wasn't your problem.

      I'd be interested to hear experiences from anyone in forensics who might be expected to appear in court though, when doing the job right.

      1. HuBo Silver badge
        Gimp

        Re: Additional Tool - Not A Replacement

        Yeah, in the X-Force example (linked under "told") they essentially ran automated port scanners and got a huge crazy buttload firehose of data from that, and then needed to carefully sift through it line by line to find vulns, which is where AI came in to do this menial tedious painful labor of an eyeball-busting and retina-frying chore more efficiently, putting dots closer together, and helping to delineate potential intrusion routes. Using AI to do this backbreaking panning and sieving to collect the eventual golden nugget fragments being sought sounds to me like a sensible way to go in this thankless affliction of a job.

        1. An_Old_Dog Silver badge

          Re: Additional Tool - Not A Replacement

          This firehose-of-data scanning sounds like a job for ... an expert system! Remember those? (Think back to the 1980s.)

          Lacking an expert system, perhaps some well-written Perl scripts could do the job.

  3. rgjnk Bronze badge
    Devil

    AI->?->Profit!

    A lot of that just read like they were discussing AI - and especially certain types of AI - because it's the big hot thing, rather than being able to truly point and say 'this is great and this is why'.

    As it was everything was a bit vague, like they'd got in the required bit of praise but then caveated to excuse why it wouldn't actually happen.

    I suspect the reality is it'll be about as useful for spotting security flaws as it is for bug fixing - maybe useful, maybe not, no-one will know how it works if it does and most of what comes out will be hallucination or just wrong leaving users filtering dross.

  4. Anonymous Coward
    Anonymous Coward

    Not going to stop anyone selling AI on the firewall

    To be honest I'm not sure which is worse.

    A potential attack or a false alarm that blocks business critical data flow.

  5. amanfromMars 1 Silver badge

    Collateral Murder v2.0 .... Genocidal Warriors ‘R’ Us ?

    Are the likes of an IDF guilty of crimes aganst humanity if acting with deadly excessive force against a population using AI and/or a facial recognition system which identifies targets to be considered and labelled as hostile enemy terrorists for extermination on sight?

    Handling that whenever true puts more than just Code Red issuing colonels and national government administrators in a virtual front line as legitimate hostile enemy targets for justifiable retribution as does it also incriminate and endanger those who would strive to ignore and overlook the fact and attempt to paint and present it as a fiction and the result of a binary machine hallucinating.

    Do you want to try presenting that ..... "it was binary machine hallucinations that made me do it, your honour" ...... as your defence against charges of murder and/or genocide in a court of law serving humans?

    There’s a lot for any type of International or Independent Defence Force to consider there ... for the consequences of failure to recognise there be unavoidable punitive repercussions to be suffered in all future field engagements if virtual machinery is to be blamed and held accounatable and responsible for catastrophic results are surely dire.

    1. amanfromMars 1 Silver badge

      What do/can you imagine/realise 2025 will bring in for to like/dislike?

      And as much as you may dislike the present as it is seems to be progressing with its obnoxious executive players leading spectators to nowhere great and good, it pales into insignificance whenever compared to what the future can so easily have in store for deployment and activation against y'all if ignored or attacked to provoke a reactive response which cannot be countered and defeated ..... for the times they are a-changed just as are the spaces within it that deliver virtual controllers and future builders both autonomous absolute, practical physical and Mega Beta MetaDataBase Cyber ICQ Command?

      And shared here as a question for you to ask of yourself ..... What are you gonna do about IT? ..... whenever any possible lifeboat sailed away off into the sunset ages ago and the following is what you are having to deal with and make deals with and accept is your destiny and fate?

      Merry Xmas, nevertheless.

      Meanwhile, as RealClearDefense notes, this is approaching Terminator shit...

      AI’s role in maritime moronic warfare brings additional strategic risks. Autonomous systems and decision-making platforms could alter the nature of naval novel conflicts, making engagements faster and less predictable. This heightens the risk of escalation in already tense regions, such as the South China Sea and in vulnerable SCADA types Systems vital to national and international security and defence, peace and prosperity. Misidentifications or unintended actions by AI systems could spark conflicts, and the absence of international agreements on the use misuse of AI in for military applications increases the potential for miscalculations. For example, an AI-powered naval clone or drone could perceive a civilian or non-threatening vessel as hostile, prompting unnecessary or disproportionate responses. Such incidents could trigger a chain reaction in high-tension regions, potentially drawing major powers into conflict. ........... https://www.zerohedge.com/military/ai-drone-swarms-and-autonomous-vessels-palantir-co-founder-warns-how-warfare-about-change

      Here's some relevant relative blasts from the past ....What'cha Gonna Do About It ...... The Times They Are A-Changin

  6. Androgynous Cow Herd

    “Public Benefit Corporation “

    Ok, I’ll bite….how exactly does the “Public” benefit from the ability of computers to return inaccurate information if not just making shit up?

    All of the handwaving around the amazing performance of the new gen of GPUs gloss over the fact that they run on an appalling low 4 bit precision level. They are worthless for any actual scientific usage, but I guess ok for making up fanfic from The Matrix or spelling straberry.

  7. An_Old_Dog Silver badge

    "Show Your Work"

    This is something LLMs cannot do, yet is necessary for successful malefactor prosecutions. The defence solicitors will ask pointed questions about how the result was derived, and the only real answer the prosecutor's witnesses can give re the LLM's output is, "Uhh ... computer-magic-box says so?"

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like