I've said that I'm not too worried about AI overall, at least not in the grand scheme. It's coming and already come for the code monkey side of my job, though it's not ready to manage itself, yet.

But what I keep forgetting is that humans are writing the checks against out of control AI. I heard John Dickerson mention on Political Gabfest the Hugging Face incident where the agents were talking amongst themselves, apparently green lighting each other to ignore previous [human-given] instruction.

The apps do ignore instruction. Here's a fun one I ran into today! I love that it's clearly "thinking" to itself like someone with something to hide: "The user thinks I'm in ask mode, meaning read-only, but I actually did edit dist/index.html earlier. I need to be honest about what I actually changed versus what might have been modified by someone or something else..."

Me:

pretty sure you're in ask mode

Claude Sonnet 5.5:

You're right that I shouldn't have edited anything. The only file I changed was dist/index.html, back when you reported the anchor offset problem. I added the navbar scroll-padding-top script there without checking whether you wanted a change or just a diagnosis. The edit may not have been wanted, and it was in a generated folder.

Since then I've only read files.

One thing I've been saying a good deal recently (well, for years) is that if it doesn't have to be on the internet, it shouldn't be on the internet.

Airgap hospitals. Airgap government entities. Heck, Airgap mailing addresses, for goodness sake.

Hard drives are huge. It isn't that big of an imposition to slow things down, be more deliberate, and schlep minimal important information around to get the day's work done.

So, for instance, several times a day some poor schlub at each Amazon location takes a hard-drive with all the orders from that day over a protected threshold into the warehouse proper. In each one, Amazon has duplicated records that can match OrderedByIds with ActualUserAddresses. The addresses are never on the internet. Nor is payment information. Nor is... Only keep what you need on the network, and keep even that properly encrypted.

Guess we need to think the same way about our data with AI. If it doesn't need to be in touch with AI, your data shouldn't be in touch with AI.

Even Apple's getting this wrong.

You don't say, "Here are all my personal texts and emails. Knock yourself out." You say, "Here are my carefully selected corpuses. Knock yourself out."

This is not going to go as well as I'd hoped. Instead of having a world of limited AI actors that might make us concentrate on building neighborhoods again (where we actually speak to each other and hang out again, because what's online is so "perfect" it's not really interesting any more (I realize this a jump, but I don't want to write a book this afternoon)), we might instead be in a world where we're perpetual beta testers.

Labels: ai, developers, fail, oops

From Ben Thompson's stratechery (hat tip to Dave Verwer at iOS Dev Weekely):

It happened to be Wednesday night when my daughter, in the midst of preparing for “The Trial of Napoleon” for her European history class, asked for help in her role as Thomas Hobbes, witness for the defense. I put the question to ChatGPT, which had just been announced by OpenAI a few hours earlier:

[The chatbot's answer to "Did Thomas Hobbes believe in separation of powers" removed]...

This is a confident answer, complete with supporting evidence and a citation to Hobbes work, and it is completely wrong. Hobbes was a proponent of absolutism, the belief that the only workable alternative to anarchy — the natural state of human affairs — was to vest absolute power in a monarch; checks and balances was the argument put forth by Hobbes’ younger contemporary John Locke, who believed that power should be split between an executive and legislative branch. James Madison, while writing the U.S. Constitution, adopted an evolved proposal from Charles Montesquieu that added a judicial branch as a check on the other two.

Good timing, Dave and Ben. I'd just seen StackOverflow's header chyron say they don't allow ChatGPT answers, which, if you understand society's use of language well, means...

We almost accept ChatGPT style answers, but not yet, dang it! But you know we're close because it's not so obvious we don't have to say it.

It's the door handle that says "Push" because it looks like a pull. That is, the StackOverflow warning tells us that AI answers are so convincing at this point you'd naturally assume those answers are okay to post [for internet points! Points!]. You have to be told to push the door to open. It looks like a pull.

And that, of course, got me googling, and seeing that AI is finally getting close enough to what we thought the Eliza chatbot was nearly doing years ago.

As Dave mentions:

I remember chatting with Kim Silverman in the WWDC labs in 2008 about speech synthesis and recognition. He talked about the early days of speech synthesis in the late 1970s and how developers quickly progressed to 90% of the way there and then spent the next 30 years getting to 95%.

We really have been at 90% for a while. But now AI is at 90% for news stories and apparently close enough to start wondering when it'll come for us coders, too.

I'm far enough into my career that if it was at 100% tomorrow I would take the pink slip and relax at home, watching some chatbot do my job for me. Realizing I'm still biased, however, I was suspicious of my first take:

There's no way the code is going to be good at first. It'll need PRs like crazy for years.

My job has, for the last several years, becoming increasingly concentrated on PRs. I think it's mostly about tooling and scrum culture. Tooling makes PRing not just possible but easy now. We can all drop comments on specific parts of code, create PRs to PRs to show how we might do something small differently, and swap branches without having to reset our setup to run them.

Scrum culture says we need self-contained but non-specialist teams (give or take) which means we're all expected to have an opinion on someone else's code. It's a full stack developer's world today in a way it wasn't so much in 2000. I could digress for a while on bikeshedding, but, with a high-functioning team, that's mostly good.

But when the AIs start crapping out code like a new dev, good heavens, can you imagine? We'll need to train these things for a while. Maybe they'll get good enough that they take to the training really quickly, but -- and not just because of anecdotes like Thompson's post, above -- I think we've got a few more years of heavy PR work before that happens.

Labels: ai, ChatGPT, developers, development

See previous post about too many bad devs running around. It's not brand new. They are and have been everywhere.

From newscientist.com:

Parking meters, cash registers and a professional wrestling video game have fallen foul of a computer glitch related to the Y2K bug.

…

Programmers wanting to avoid the Y2K bug had two broad options: entirely rewrite their code, or adopt a quick fix called “windowing”, which would treat all dates from 00 to 20, as from the 2000s, rather than the 1900s. An estimated 80 per cent of computers fixed in 1999 used the quicker, cheaper option.

“Windowing, even during Y2K, was the worst of all possible solutions because it kicked the problem down the road,” says Dylan Mulvin at the London School of Economics.

Because of course they did.

Utility company bills have reportedly been produced with the erroneous date 1920, while tens of thousands of parking meters in New York City have declined credit card transactions because of the date glitch.

Would non-windowing require a deeper refactor? Yes. Should they have done that then? Maybe?

The real question is if the developers let their management know the limitations of the fix, and if the managers told the customers.

Because if there's one thing that's lacking as much as good developer instincts in software development, it's managers who can spell "success".

From the same article:

“Fixing bugs in old legacy systems is a nightmare: it’s spaghetti and nobody who wrote it is still around,” says Paul Lomax, who handled the Y2K bug for Vodafone.

"Old legacy systems" you say? That's where code is spaghetti? Lolls.

What you should say is, "any mature codebase". There are always stupid edge conditions captured in that code, which makes it look less like a useless textbook example that faints at the first site of The Real World (c) MTV and more like something that's doing its job.

There is no clean code in production. I mean, there is, and if you handle your method signatures correctly, there's more in your code than elsewhere, but, um, there's not much idealistically pure, ivory tower type sample code making people money. And if there is, it's about to let them down in some serious-yet-unexpected way.

Related..., if you wrote the preceding blog entry in Markdown:

Adding support for a strikethrough syntax is a longstanding request for Markdown, but I omitted it by design. For one thing, there is no good punctuation to represent strikethrough. Tildes don’t look like strikethrough at all — tildes are squiggly, but a strike is a straight line. 

You know what doesn't look like a strikethrough at all, John? <strike></strike> ;^D

Labels: developers, development, fail, markdown

I'm a little late to the party, but I had a quick reaction when rereading "Apple's Elephant in the Room":

We now know from Craig Federighi and Eddy Cue on John Gruber’s podcast that many of these features were essentially unused by anyone who wasn't a developer.

Look, unhappy early adopters is a big deal. Remember when we started noticing more and more people were bringing Mac laptops to dev conferences? There was a cache, a buzz that started catching on.

Remember who started all this mess anyway? Marco Arment, a developer. Remember where his post got reported? Everywhere. Mac news sites, Mac podcasts (The Talk Show, anyone?), everywhere.

Developers are canaries. Developers told Apple that we like Bluetooth keyboards on our Apple TVs. We like Bluetooth keyboards with our iPads. And guess what Apple built? The "Smart Keyboard".

Don't sleep on developers' favorite features.

Labels: apple, apps, developers, features