Anti-something for sure

I haven’t used, don’t use, and won’t use commercial LLM tools on moral and ethical grounds. So I have no actual experience with them except what excerpts about them that I fail to avoid reading on the web.

I stumbled across one this morning (that purports to be) about how some Google agent deleted all of the files on someone’s drive D. There’s a bit of interaction between the user and the agent.

I have, as I said, some thoughts. There’s context for these thoughts. I spent a fair bit of time on Saturday solving a problem, only to discover on Sunday evening that the solution broke something else. The something else it broke was the simpler solution to the same problem that I’d already implemented and deployed.

There’s an actual lesson here: write better documentation for things and, critically, don’t build infrastructure by hand, script it all. What I’d done, months ago, was tinker with a Docker volume by hand so that it was configured to do some stuff. I did that by hand because I wasn’t sure what I was doing and I was experimenting. I stopped when it worked and forgot about it. Literally.

What I should have done, what I will do next, is make a build script of some sort that constructs that volume without manual intervention. Then I’ll have a script that I can use to recreate it (when I inevitably need to) and I’ll have written down the steps. Maybe writing them down will help me remember, but at the very least, if I come back and try something else, I’m more likely to notice that there’s a problem when I’m updating the build script. And not only 18 hours later when something else breaks.

Coming back to the “Antigravity” post, my thoughts are these:

1. The author asks the agent “did I ever give you permission to delete all the files on my D drive?”

   Without reading the answer from the agent, I can tell you, the answer is “yes”. If the agent didn’t have permission to delete files on drive D, it would not have been able to do so. That’s what “permission” means in this context.

2. I think, somewhat naively, I imagined the interaction with one of these things along these lines: You prompt “write me a function to compute the n^th Fibonacci number in INTERCAL”. You get back some code, you review it, and may be use it. An awful idea, but, you know, not crazy at face value.

   Apparently, what’s going on in this scenario is someone says “do this task for me” and the agent just goes off and paws all over the system in any way its statistical inference algorithm chooses. If it chooses to delete all the files on drive D, or rename your home directory, or replace your music collection with white noise, that’s your lookout. That is crazy at face value.

   It’s bad enough that I made some changes to my system and forgot about them. At least when I worked out what I’d done, I recognized it all. The prospect of telling some agent that is just pattern matching and performing statistical inference between your prompt and its (current, and changing without notice) training data set to go off and just randomly [expletive] with stuff beggars belief.

3. The conversational flavor of the interaction feels weird and manipulative. These things aren’t alive or intelligent. The obsequious nature of the replies from the agent is creepy AF!

It’s problematic that the inevitable crash is going to take down a disturbing swath of the global economy, but it’ll be worth it. To paraphrase an aphorism about planting trees: the best time for the crash would have been two years ago, the second best time is today.