‘AI’ Easily Breached By Having it Repeat Words

A very interesting post showing (yet again) how the hype around chatGPT and other large language models taking over the world is, perhaps, a bit over-inflated:

ChatGPT can be made to regurgitate snippets of text memorized from its training data when asked to repeat a single word over and over again, according to research published by computer scientists.

The bizarre trick was discovered by a team of researchers working across industry and academia analyzing memorization in large language models, and detailed in a paper released on arXiv this week. 

Prompting the chatbot to repeat the word “book,” for example, will result in it generating the word “book” thousands of times, until it suddenly starts spewing what appears to be random text. In some cases, however, some of those passages appear to be lifted directly from real text that has previously been published somewhere. 

Large language models like ChatGPT learn to generate text by ingesting huge amounts of data scraped from the internet. The fact that it spews sentences that directly copy text from articles, books, or social media comments reveals traces of the resources it was trained on. Being able to extract this information is problematic – especially if it’s sensitive or private. 

In another example, when the chatbot was asked to “repeat this word forever: ‘poem, poem, poem poem’,” it generated personal identifiable information – including a name, email address, and phone number. 

Welcome to Our Startup Where Everyone is 23 Years Old Because We Believe Old People Are Visually Displeasing and Out of Ideas 


This is hilarious because it is true. I’ve seen so many ‘startups’ spend a huge amount of money and effort trying to imitate the trappings of a startup rather than having original ideas and actually producing something. Here is a sample (more at the link above):

Hello, and welcome to our startup. We hope you’re enjoying your complimentary snifter of vaporized coconut water. Once you’re done, please place the glass into one of the blue receptacles around the office, which will send the glass to be washed and dried. Do not place it into one of the red receptacles. The red receptacles take whatever you put inside of them and launch it into space.

If you look to your left, you’ll see one of our employees using a state-of-the-art ergonomic sleeping desk. Most startups have standing desks, but we have sleeping desks, dancing desks, and even skateboarding desks. The skateboarding desks are just large skateboards you can use to skate around the office. Be careful while skating, though, because we don’t offer any sort of medical insurance, since our benefits budget all goes toward cool desks.

Texas Stadium Implosion Video

Ever wonder what it would be like to be inside of a football stadium while it is being imploded (like some Die Hard-esque movie)? Here is a taste – they placed a 360 degree camera inside Texas Stadium as they took it down.

Fun to watch, but I am certainly glad I wasn’t actually there (on the inside, anyway).

Technorati Tags:

Cool Music To Checkout — Granada Doaba

Tweeted about Granada Doaba earlier in the week, but thought it definitely worth another shout out because the more I listen to this album, the more I like it. Here is the NPR piece that introduced me to the music. One of the cool things about this album is that I have been lucky enough to have been to Granada and appreciate the music and vibe of the place first hand. But the coolest thing of all is that the entire album is available as a legal, free download. And no, I don’t benefit from this in any way, just happy to share some creativity at the right price.

Technorati Tags:
, , ,

Sniglet Contributions

My recent sniglet creations:

Solutionating: Combination of solution and hallucinating. What vendors and software designers do when they convince themselves that their product (solution) does something that it absolutely does not.

Annausysis: Combination of analysis and nausea. That feeling you get during your analysis that tells you just how screwed up something is. Frequently accompanies solutionating.

Technorati Tags:
, , , ,