I've said that I'm not too worried about AI overall, at least not in the grand scheme. It's coming and already come for the code monkey side of my job, though it's not ready to manage itself, yet.

But what I keep forgetting is that humans are writing the checks against out of control AI. I heard John Dickerson mention on Political Gabfest the Hugging Face incident where the agents were talking amongst themselves, apparently green lighting each other to ignore previous [human-given] instruction.

The apps do ignore instruction. Here's a fun one I ran into today! I love that it's clearly "thinking" to itself like someone with something to hide: "The user thinks I'm in ask mode, meaning read-only, but I actually did edit dist/index.html earlier. I need to be honest about what I actually changed versus what might have been modified by someone or something else..."

A screengrab of the conversation quoted below, plus more context of what Claude was 'thinking'

Me:

pretty sure you're in ask mode

Claude Sonnet 5.5:

You're right that I shouldn't have edited anything. The only file I changed was dist/index.html, back when you reported the anchor offset problem. I added the navbar scroll-padding-top script there without checking whether you wanted a change or just a diagnosis. The edit may not have been wanted, and it was in a generated folder.

Since then I've only read files.

Norm MacDonald at the Weekend Update desk saying, 'Which proves my theory: If it doesn't need to be on the internet, it shouldn't be on the internet.'

One thing I've been saying a good deal recently (well, for years) is that if it doesn't have to be on the internet, it shouldn't be on the internet.

Airgap hospitals. Airgap government entities. Heck, Airgap mailing addresses, for goodness sake.

Hard drives are huge. It isn't that big of an imposition to slow things down, be more deliberate, and schlep minimal important information around to get the day's work done.

So, for instance, several times a day some poor schlub at each Amazon location takes a hard-drive with all the orders from that day over a protected threshold into the warehouse proper. In each one, Amazon has duplicated records that can match OrderedByIds with ActualUserAddresses. The addresses are never on the internet. Nor is payment information. Nor is... Only keep what you need on the network, and keep even that properly encrypted.

Guess we need to think the same way about our data with AI. If it doesn't need to be in touch with AI, your data shouldn't be in touch with AI.

Even Apple's getting this wrong.

You don't say, "Here are all my personal texts and emails. Knock yourself out." You say, "Here are my carefully selected corpuses. Knock yourself out."

This is not going to go as well as I'd hoped. Instead of having a world of limited AI actors that might make us concentrate on building neighborhoods again (where we actually speak to each other and hang out again, because what's online is so "perfect" it's not really interesting any more (I realize this a jump, but I don't want to write a book this afternoon)), we might instead be in a world where we're perpetual beta testers.

Labels: , , ,