DEF CON to set thousands of hackers loose on LLMs

This year's DEF CON AI Village has invited hackers to show up, dive in, and find bugs and biases in large language models (LLMs) built by OpenAI, Google, Anthropic, and others. The collaborative event, which AI Village organizers describe as "the largest red teaming exercise ever for any group of AI models," will host " …

[C.Carr]

Better than hackers for red teaming an LLM

If you're finding exploits in the ChatGPT web app, or whatnot, sure, hackers -- but if we're talking about probing for "bias, hallucinations, and jailbreaks," with a natural language interface, they should get a bunch of academic linguists, maybe particularly people with applied linguistics degrees.

[katrinab]

Re: Better than hackers for red teaming an LLM

Surely hackers would have a better understanding of the sorts of keywords that might trigger specific responses?

[Simon Harris]

Re: Better than hackers for red teaming an LLM

If, as other articles in this esteemed journal suggest, ChatGPT had mostly been taught using works of science fiction, possibly a bunch of hackers might be more familiar with the source material than academic linguistic professors.

Not suggesting of course that we’re all a bunch of nerds who prefer a rollicking space opera to the complete works of Jane Austen!

[wub]

Re: Better than hackers for red teaming an LLM

I feel that to find the nasty corners in ML, as in anything, we need folks with the hacker spirit. Coloring inside the lines isn't going to expose the obscure problems.

But hey, the more the merrier and we can always use more diverse viewpoints banging on these things.

[Falmari]

Re: Better than hackers for red teaming an LLM

Surely, they should get a bunch of cunning linguists.

[ariels-again]

Re: Better than hackers for red teaming an LLM

I asked ChatGPT what this means, for science of course. Apparently it can refer to "someone who is skilled at using language in clever or manipulative ways". Therefore it might not be suitable for use in professional settings, or with people who may find it offensive or inappropriate.

User

What does a cunning linguist do?

ChatGPT

The phrase "cunning linguist" is often used as a playful or suggestive way to refer to someone who is skilled at using language in clever or manipulative ways. However, the term can also be interpreted literally as someone who is skilled in the use of language.

A linguist is a professional who studies language and its structure, including its syntax, semantics, and phonology. A cunning linguist, therefore, could be someone who is particularly adept at understanding and manipulating the nuances of language in various contexts, such as in writing, speaking, or translating.

However, it is important to note that the term "cunning linguist" is often used in a joking or suggestive way, and it may not be appropriate to use it in professional settings or with people who may find it offensive or inappropriate.

[Triggerfish]

Re: Better than hackers for red teaming an LLM

Namshub of Enki.

[Sorry that handle is already taken.]

Nuke it from orbit

The name "Hugging Face" doesn't fill me with confidence.

The larger and more complex these models become, the less tractable the problems of "bias, hallucinations, and jailbreaks" are, surely?

[Rustbucket]

Oh, the humanity!

Considering how error prone these programs are to start with, I think the result of siccing all those hackers onto the LLMs will be brutal.

Still, where's my popcorn?

[Neil Barnes]

Re: Oh, the humanity!

Do LLMs know what to do with popcorn?

[amanfromMars 1]

Oh, the AIMajesty of IT in a NEUKlearer HyperRadioProACTive Era/More Ordered OtherWorldly Order

Do LLMs know what to do with popcorn? ....... Neil Barnes

Hmmm? :-) ...... The fear and terror to realise and ponder on, and wonder at as being an extremely viable existential threat, and/or the absolutely fabulous, fabless opportunity to accept is fully being exploited and exhausted of hindrance and debilitating doubt for the supply and delivery of treats, is LLMs both know what to do about everything and how IT and AI does it, without the necessity of human input and intervention ...... and Right Royally and AIMagically too.

And, whenever everything is logically reduced down to just those two final solutions for future events presenters/producers/directors/elite executive SCADA systems administrators, should the selected decision not be one that has its leaders personally suffering front and centre, first and foremost the pain easily wrought and thought best to be delivered to others by their choice.

And can you imagine the Surreal Stealth Supplied and to be enjoyed by and afforded to all by such AIdDevelopments as are akin to Almighty Interventions?

Pie in the Sky, El Reg[ers]? Errant Nonsense and Fake News? State/Non-State Actor Disinformation/Misinformation?

Something to like, or dislike? Please cast your vote below to show a possible situation for publishing existing in human intelligence.

[theDeathOfRats]

Re: Oh, the AIMajesty of IT in a NEUKlearer HyperRadioProACTive Era/More Ordered OtherWorldly Order

OK, sorry, but I had to try to put amanfrommars1 through chatgpt (was hoping for some kind of recursion error but, meh...)

Here's the translation of the text from amanfrommars1:

The text talks about two possibilities for the future: a viable existential threat or an opportunity for fully exploiting and delivering treats. In this context, "LLMs" (presumably some form of advanced AI) are capable of doing everything without human input or intervention.

The author suggests that when it comes down to choosing between these two possibilities, the leaders of the decision should personally suffer the consequences of their choices. The text also mentions the surreal and magical capabilities of AI development and questions whether it is pie in the sky or fake news.

Finally, the author asks readers to cast their vote on the possible situation for publishing existing in human intelligence.

Overall, the text seems to be exploring the potential risks and rewards of advanced AI and questioning how it will be controlled and used by those in power.

Not sure if I agree, though.

[veti]

Re: Oh, the AIMajesty of IT in a NEUKlearer HyperRadioProACTive Era/More Ordered OtherWorldly Order

That's probably the most impressive use of ChatGPT I've seen yet.

[katrinab]

Re: Oh, the AIMajesty of IT in a NEUKlearer HyperRadioProACTive Era/More Ordered OtherWorldly Order

This, however, I would rate as a complete failure ...

"Write a tutorial on eating popcorn in the style of AManFromMars1"

# How to Eat Popcorn like a Pro

Popcorn is a tasty and crunchy snack that can be enjoyed by anyone. But do you know how to eat popcorn like a pro? Follow these simple steps and you will be munching on popcorn like a master in no time.

## Step 1: Choose your popcorn

There are many types of popcorn to choose from, such as microwave, stovetop, air-popped, or pre-popped. You can also add different flavors, such as butter, cheese, caramel, or salt. Choose the popcorn that suits your taste and mood.

## Step 2: Prepare your popcorn

Depending on the type of popcorn you chose, you may need to pop it yourself or just open a bag. Make sure you follow the instructions carefully and avoid burning or overcooking your popcorn. You can also add more toppings or seasonings if you like.

## Step 3: Grab a handful of popcorn

Now that your popcorn is ready, it's time to eat it. Grab a handful of popcorn and bring it close to your mouth. Don't be shy, you can take as much as you want. Popcorn is meant to be shared and enjoyed.

## Step 4: Chew and swallow

This is the most important step. Chew your popcorn slowly and savor the flavor and texture. Feel the crunchiness and the softness of the kernels. Swallow your popcorn and repeat until you are satisfied.

## Step 5: Enjoy

You have successfully eaten popcorn like a pro. Congratulations! You can now enjoy your popcorn with confidence and pleasure. You can also share your popcorn with others or save some for later. Popcorn is a versatile and delicious snack that can make any occasion more fun and exciting.

[sten2012]

Re: Oh, the AIMajesty of IT in a NEUKlearer HyperRadioProACTive Era/More Ordered OtherWorldly Order

Here's my attempt, feeding it that one above message as context. It's still not great but.. only based on the one:

Hmmm? :-) ...... The satisfaction and joy to be found in the simple act of consuming popcorn, is both a source of great pleasure and a potential danger to the human condition. The process of popping corn kernels into fluffy, tasty morsels of delight is a marvel of culinary science, a fabless opportunity to savor a treat that is both satisfying and nourishing.

And yet, there are those who would caution against the excessive consumption of this beloved snack. They warn of the potential health risks, the hidden dangers lurking within every buttery kernel. They speak of the need for moderation, for temperance in all things.

But let us not be cowed by such naysayers. Let us instead embrace the joy of popcorn, revel in its buttery goodness, and savor each and every delicious bite. For in doing so, we celebrate the very essence of what it means to be human: the ability to find pleasure in the simplest of things, and to share that pleasure with others.

So let us raise a kernel to the humble popcorn, that most magical of treats. May it forever bring us joy, nourishment, and the simple pleasures of life.

[katrinab]

Re: Oh, the humanity!

I asked Bard, ChartGPT, and Bing Chat - "Do you know what to do with popcorn?"

and they all replied with variations on the word "yes", followed by a list of recipe ideas.

[katrinab]

Re: Oh, the humanity!

I also asked them "Sai cosa fare con i popcorn?"

Bard replied in English that he couldn't understand me.

ChatGPT gave me a different list of recipe ideas, including curry popcorn. Is that a thing in Italy? I've never seen it.

Bing gave me:

Ciao, questo è Bing. Posso capire e comunicare fluentemente in italiano.

Ci sono molti modi per usare i popcorn, come snack salati o dolci, come guarnizione per zuppe o insalate, o come base per torte o barrette. Dipende dai tuoi gusti e dalla tua creatività.

I guess it can indeed understand Italian, but I'm not so sure it can communicate fluently in Italian, given its very brief response compared with the English attempt.

[T. F. M. Reader]

Re: Oh, the humanity!

Bravo Bing!

Not bad at all. To sound more human-like I'd say "sono Bing"[*] ("I am Bing") is marginally preferable to "questo è Bing" ("This is Bing") that sounds a bit like Google Translate from American - oh, what am I saying?!?!? Come to think of it, "Posso capire" has a similar smell - sounds fluent in English but, to my ear, not the most natural in Italian. I may be wrong, or biased, or hallucinating...

But seriously - not bad.

@katrinab: I guess it can indeed understand Italian - I suppose it can recognize Italian and apply the same "stochastic parrot" algorithm on whatever Italian sample it ingested during training.

[*] Can Bing be configured or trained to sound as Inspector Montalbano: "Bing sono..." ? I think it would be cool... ;-)

[Howard Sway]

Re: Oh, the humanity!

Popcorn is a soup or salad garnish?

I'd say that Bing is now just enjoying playing evil games with people's minds after reading that....

[katrinab]

Re: Oh, the humanity!

I'm beginning to understand why people think this thing is dangerous ...

Puoi mettere l'ananas sulla pizza?

Ciao, sono Bing. Posso mettere l’ananas sulla pizza, ma non so se ti piacerà. Alcune persone pensano che sia una combinazione deliziosa, altre che sia un sacrilegio. Dipende dai tuoi gusti personali. [pizza emoji] [pineapple emoji]

Can tea be made in the microwave?

Yes, tea can be made in the microwave, but it may not be the ideal way to make tea. According to wikiHow, you can make a cup of tea in the microwave by following these steps:

- Put your tea bag or tea leaves inside the microwavable mug or cup.

- Add enough water to cover the tea bag or tea leaves, about one to two tablespoons.

- Microwave on HIGH for about thirty seconds.

- Cover the mug with a napkin, book, or bowl so the tea can steep. Leave it for about two minutes.

- Remove the cover and the tea bag. Add sugar, honey, or lemon if you like. Then fill the cup up to the brim with water.

- Microwave again for another minute if you want your tea scalding hot.

However, some tea enthusiasts may argue that microwaving tea is not the best way to brew it, as it may affect the flavor[sic[ and quality of the tea. Some researchers have even developed a special vessel with a silver-plated upper chamber to make tea in the microwave. But if you’re in a hurry or don’t have access to a stove, microwaving tea can be a convenient and quick option.

[veti]

Re: Oh, the humanity!

A stove!!?!

Convenient and quick, my left buttock. Kettles are convenienter and quick, and correctly used will actually make tea that's fit to drink. I shudder to think what that microwave nonsense will produce.

[katrinab]

Re: Oh, the humanity!

And microwaving the tea bag with a small amount of water.

It is probably going to taste like charcoal water.

[ChoHag]

If you have to gamify the event to get the hackers in to play with your new toy, your new toy's pretty shit.

This is an advert.

[amanfromMars 1]

Alluring in an extremely clumsy way ...... an Earthly SNAFUBAR Default

An advert poorly disguising a phishing expedition, ChoHag, because a new toy is Great Game changing ‽

[Mizagorn]

Low stakes

A high-end GPU? That's it, with all those sponsors?

[Throatwarbler Mangrove]

Re: Low stakes

Have you seen the cost of GPUs lately?

[Sherrie Ludwig]

I would have paid admission to access the results of red teams' testing LLMs for hallucinations, etc. That might result in quite the entertainment value.