Fixing stuff by driving an LLM to do it seems like a distinct skill unto itself. It’s kind of this blend of your normal debugging skills, carefully managing what goes into the LLM and carefully constructing your requests.
For me at least it’s the one thing that feels like a novel new skill to learn more than anything else. Normal LLM prompting feels like just an extension of the mental process I’d use when coding and designing in a way that trying to get the LLM to fix things kind of doesn’t.
I agree that it is a bit weird, kind of coding in natural language but also not exactly like that. Maybe as the agents stabilize and there are proper new tools for this new process we can relax a bit and it will become the new normal. For now it feels like surfing over an ever changing seascape, exciting but tiring at the same time.
If I write the song lyrics myself, does it become valid art then?
If I listen to it and then use that feedback to adjust the song structure, because it’s not giving me the feeling that I want to convey to others, is it art?
If I then repeat that multiple times, is it art?
What if I then record my own guitar parts or vocals, what then?
I don’t necessarily think these things can have an answer but I outright disagree that “if AI is used then the thing that’s made is categorically not art”.
I think it was Dr K of Healtgygamer that explained it something like non-toxic shame is a useful signal — “I did something wrong and this is a signal telling me so; I need to correct my behaviour” vs toxic shame as in “I am fundamentally a bad/worthless person”. We kinda got the two tangled together and in trying to reduce the second, we probably demonised the first a bit too much.
John Bradshaw defined toxic shame as distinct from a healthy sense of dignity, manners, and modesty. Also, he addresses how toxic shame can be "carried" by people, and passed down in families, as a legacy, so that eventually those who are afflicted need to heal from it, to stop the cycle.
I have personally read this book (found it in community college library) and I found myself wishing I'd paid more attention to his talks while he was active.
That's a great point. And it goes probably both ways. The young person doing something annoying might also be annoyed by something different the older person does. Respectful communication is probably the key but why is it so elusive ?
I think this is something we’ll see AI very meaningfully impact — the threshold needed for “Get this thing working on a tech stack we might not be familiar with” has gone waaaaaaay down.
It's worrying to say the least. Today it's "get this working", tomorrow it's "can it also do XYZ?", next week it's "we have a 40% spike in crashes, you MUST resolve this IMMEDIATELY!"
Meanwhile the devs are furiously asking AI how to fix it, every flavor of every model will give you a different diagnosis, GH Copilot will throw a million high/critical at you, and you still have no idea if it's fixed or not.
It's the exact reason why you still need to know the languages you're using despite what leadership/product teams demand.
Yeah. It’s also far too tempting and far too easy to keep adding features if you don’t make yourself disciplined about it (which is okay if you’re in a position to control that, not so good if it’s the higher-ups demanding it).
But that's a lot more words than "I found these issues". I also don't particularly care that 'Tool x ran for x amount of time', I don't want it re-telling me that -- I've probably been sitting there watching it do that, so I already know.
Then again, I just dont have an issue with the 'I-isms'; it's a bit weird sure, but at the same time it's a bit more pleasant to interact with as well. After all it is trying to model itself as a person you're talking to.
I have fond memories from back when Pixels were actually flagship. The Pixel 2 is my nostalgic pinnacle. I'd totally forgotten about that squeeze gesture until I got my old Pixel 2 out to erase.
I've found that LLMs are specifically bad at a certain kind of debugging, though I can't quite put my finger on what that is.
"Spot the bug in this code" when the code can be looked at and pattern-matched against bugginess is something they seem really good at.
Some parts of debugging, like "Here is this logfile, what do you think is going on?" are also surprisingly good.
It's that thing kind of in the middle -- I know it when I see it honestly is the best way I can put it into words. An example from recently, I'm receiving some bad data on a network message parser. Immediately I don't know whether it's a my-side or their-side thing, but I know if I try and just vaguely describe the behaviour to the LLM it will start churning tokens.
My current approach to problems like this is -- I need to tell the LLM what it needs to do to give itself the data it needs to solve the problem. My first reaction now isn't "It's not working, there's a bug, it's not doing X". It's "Okay, this isn't quite working properly; I need you to add some debug logging around X, Y and Z so we can figure this out". That tends to avoid spirals and get me out of the situation much more quickly.
The seeing eye dog analogy is pretty apt actually. I would love to see some transcripts from the author if they are able.
Edit to add: I think the 'thing' I'm alluding to might be -- if I have trouble expressing the buggy behaviour clearly in words, then I know it's probably going to be a fair few back-and-forths with the LLM to get something; the harder I find it to concisely describe, the more risk that it'll fall into a pit. Doubly so if I offer up a hypothesis which turns out to be wrong.
“Model X CHANGES EVERYTHING!” (surprised Oculus Rift face without the headset)
Some one-shot prompts for quick impact.
I get it with the incentive structure — honestly not trying to shit on any creators (hate the game, not the player) but seeing it every day is just getting tiring.
I have some memories that I know for a fact are wrong in a pretty particular way. They’ve either become attached to the wrong place or the wrong time.
Like I remember a good friend picking me up from my childhood home, yet I didn’t even know that person until I’d long moved out (many years) from that place.
For me at least it’s the one thing that feels like a novel new skill to learn more than anything else. Normal LLM prompting feels like just an extension of the mental process I’d use when coding and designing in a way that trying to get the LLM to fix things kind of doesn’t.
reply