It's the uncanny valley of AI. It's still not quite good enough yet that you can trust it blindly on a big codebase, so you still have to read and understand everything - which is often harder than just writing it up yourself.
EDIT: don’t get me wrong. I still think AI is incredibly useful for a lot of tasks! But when implementing an architecturally hairy thing, I find it less stressful and equally quick to jump down to the editor level and use AI just for code completion.
Pretty much, The one thing I use it for is as a sanity check, pretty much "Look at <SomeFile>, point out issues you see, summarise them tersely" and it'll spot stuff a code review by a human might have spotted (in the mythical land where people actually do code reviews properly and don't just flag a spelling mistake to "show they looked at it").
Beyond that I don't trust it at all and I still write all my code the meat sack way.
Trust is earned not given and it hasn't earned it yet.
If anything, I think this hype cycle is fast exposing just how many people, teams, and companies just don't care about what is correct. They just wanna feel good about themselves and get paid.
I for one welcome the fact that this whole thing has driven me back to books and deeper into the fundamentals. I have never read so much on math, hardware, and history as in the past 3 years or so.
Are you doing those things for your own enjoyment though, or to eventually capitalize on it?
And if it’s just for enjoyment, then doesn’t it make sense other people who want the same would just get a job where they can keep pushing things to an AI, feel good, get paid, then quickly get back to the hobbies they really love?
I don't think "compartmentalization" does this topic justice. Note, I'm not accusing you, since I don't know your work. My post is about the general milieu and the rhetoric around it. This "compartment" terminology seems to whitewash a dimension that has diligence on one end, and fraud on the other.
I've always worked in high-trust organizations where we depended on each other operating on the diligent side of things, making conservative choices to never wander into the murky area in between. It is horrifying to me how many people seem complacent about or even complicit in a different objective, which seems hell bent on wandering as far into the murk as one can without being caught.
When the person who has a duty of diligence starts rubber-stamping AI outputs, they're veering off into that same murk. They accidentally or wantonly trust the agent as if they have delegated their duty of care. But the AI tool has no such duty and no capacity to care.
I think this worker who has turned themselves into an outsourcing middleman needs to treat the results just like "found code" in a USB drive they found in the parking lot. Its origins and purpose are unclear. It could be flawed or obscurely inappropriate for the intended application, it could have legal entanglements, or it could even be subtly adversarial. The review task to figure this out is not simple. It is not something you do by skimming the result, or worse, asking some other AI tool to review and summarize.
The person importing such code to a project needs a different kind of diligence to try to screen it. For a lot of people, I think this review may be impossible or at least no less laborious than doing the original work themselves with the required diligence.
And, I think this importer needs to be fully liable and responsible for the outcome. But, instead, I think we're seeing frequent appeals to blame the machine and act like it is an honest mistake to let things pass because they've been rubber-stamping the imports. A lazy desire to claim credit for appearance of success, but shirk responsibility for detected failures.
Software Engineers have historically not been very liable even for errors they create by their own hands, and companies tend to operate on a “blameless” engineering culture where there is not really a formal consequence for screwing something up. Your peers may not trust you for a bit, but the trust builds back rapidly as they make their own mistakes and then empathize with your past transgressions.
And in the AI era, this attitude has extended even toward code engineers aren’t writing themselves. Everyone agrees not to protest too loudly if someone commits something wrong because tomorrow it might be you who has done so. The days of “obviously bad” slop code are largely behind us and most recent LLMs tend to put out decent code, so it is very hard to catch problems in review phases, you’d have to be very eagle eyed and that’s not scalable to every employee. It is much easier to let the code run wild for a bit and just be ready to fight fires should they come up.
My advice is to try letting the agent fill in the gaps. You can probably architect better than it can. Write your class outlines, explicitly define the public facing bits and what you want APIs to look like. Write the key integration tests that you know ought to pass.
The real advantage is that agents routinely write code without any silly copy/paste mistakes like accidentally accessing x twice on a coordinate operation instead of x and y. You can add some comments for what the function should do, throw in some real/pseudo code and let the LLM figure it out.
> It's the uncanny valley of AI. It's still not quite good enough yet that you can trust it blindly on a big codebase, so you still have to read and understand everything - which is often harder than just writing it up yourself.
You might as well have left it with: "It's still not quite good enough yet that you can trust it". That's the core of the issue. It doesn't matter what you ask it to do, it can't be trusted. Some things are just easier to verify and correct than others.
I was investigating an ansible playbook yesterday that had a 45 line comment to explain a single apt install command, completely and utterly useless. I am updating my neovim to just collapse all comments, the noise is unbearable.
Limit the amount of comments you put in the code to a strict minimum. You should almost never add comments, except sometimes on non-trivial code, function definitions if the arguments aren't self-explanatory, and class definitions and their members.
Do not remove existing comments unless they are directly related to what you are changing.
Is anybody else concerned that we've gone from exactly specifying to a computer, using an exact language, what to do, to a more sophisticated form of prayer?
Nah, I'm sure it's fine. What could possibly go wrong with abandoning nearly a hundred years of CS knowledge and hard-won lessons and replacing all of it with the equivalent of trying to convince a sociopathic teenager to clean their room?
This seems to misunderstand the parent comment. The sibling comment plays along with the analogy and mentions having to fix the mistakes of an automatic chisel, but the original comment is alluding to the fact that it is definitely not an automatic chisel. There is no uncanny valley in an automatic chisel.
Programming up until this point was done by using deterministic tools to build products. LLMs appear to be nondeterministic tools in their current incarnation, at least to humans.
If an automatic chisel had a feature that could switch from chiseling from sedimentary rock to chiseling marble but would randomly and nondeterministically switch to the other mode during its use it would be considered defective. But with LLMs the industry has collectively decided that the nondeterministic automatic chisel builds so fast that the current defect rate from the nondeterminism is acceptable.
It would make sense to isolate the last line of defense from LLMs, ie the tests, but this rarely seems to happen any more. Once the tests are contaminated with LLMs all bets are off.
And people forget that along with atrophying skills and reasoning due to less coding, the skill degradation is hastened because the programmer is faced with the reality that they would have to constantly figure out, review or edit someone else's code (ie, the LLM's) if they truly wanted to maintain a last line of defense. But as this type of task is literally the least liked task in programming, the programmer passes it off to the LLM as well to avoid burnout...
And of course, Google releases another dynamite piece of information to the world, with 'recursive dreaming and feedback loops'.
This revolves around an idea that started AI, 'Cybernetics'. Feedback loops that steer and guide based on input and output. The key to autonomy.
Google's whitepaper: 'Attention Is All You Need' was another dynamite piece of info that ignited and revealed something important about Intelligence: Attention is required.
This should push the edge along, making AI even more useful to us. Attention, plus a loop to observe, reflect, is the key to autonomous agency, whose system is governed by physical processes: speed of churning information and transmitting it to where it needs to be in order to make real whatever is it you need.
So really, the key is Attention (for Understanding), Time (to do Work), and Observation (to Improve). With these three things, the outer loop can steer itself (the aware 'thing', agency).
At that point, things get weird because there is no 'outer loop' anymore. The 'thing' becomes alive and is its own outer loop: a key idea of Cybernetics. The only thing affecting performance would be the speed of processing information and transmitting it where it needs to be.
And then: ensuring adequate quality of machinery, robotics, with minimal 'environmental issues', to help reduce churn (another feedback loop that would correct a robot's errors).
The outer loops decides, observes action, improves, re-observes, improves again. Rinse and repeat. What becomes important is Persistence.
For us, it's basic survival and emotional needs to some certain level of contentment. For a Machine, it's likely simply to be: to be useful.
Completely dependent upon us. The issue is Time, and how to fairly compensate everyone.
They say writing engages more of the brain and helps us to remember what's written more than if we just read it, or copy and paste.
When you say it's easier to go manual, it seems you're talking about learning retention. And you're right.
But seniors have learned enough that they're able to iterate quickly with AI.
They know how to organize their work, manage change, tasks. They know how to break a problem down into smaller pieces. They're aware of context windows, token cost, estimated task lengths, etc. And most importantly, and to your point about ease: they have less to learn so retention isn't an issue.
I have no opinion about whether we're in a good or bad situation, just making arguments from the toilet really.
It's not just learning is it, why do people buy hand ground coffee when there quite literally isn't any difference? Or audiophile snake oil? As long as humans are still the consumers, some part of consumption will be emotional. Could be to support local artisans, could be gullibility, could be love, whatever.
Maybe one day artisanal code will be a thing lol. Hand written like calligraphy. Those with refined tastes will have their favorite code artisans. And the plebs can continue with mass produced industrial junk.
It happens, but it's rare. When last did a product take over a market without 100s of millions, sometimes even billions, of VC dollars?
There is no motivation to build a better mousetrap today, because the drooling idiot with a Claude account will look at how quick you signup users, clone it in a week (hey, it only needs to be superficially the same), and get VC money to dump until you go out of business.
I think what's missing is fundamental. I think the reason it sucks so much to work with LLM-generated code is that LLMs will never "know" what it's like to be human. They don't "understand" our frustrations and motivations, and they're missing the vast array of useful mental tactics we've evolved to cope with corporal existence.
At this point I think progress towards a good colleague bot would require a new architecture which allows continuous leaning, and for the LLM to be raised as a human child (maybe in a simulation at 1000x speed or something).
Your assumption is that LLMs will ever leave this uncanny valley.
Maybe unforeseen breakthroughs and different architectures are achieved. Given LLM fundamental shortcomings grounded in mathematics and information theory, I highly doubt they will and we will always need to deal with these issues in some capacity.
Agreed. For AI to be something you can trust to operate autonomously, it needs to actually be able to understand the things it is working with and reason about them. LLMs cannot, by their very nature, do that. There can be no reliability with such a tool.
I think that bigger context windows help, but I feel that for AI to cross this chasm, it needs to be able to encode more abstract context knowledge. I think the holy grail here is online learning.
I think the speed/context size of the large models is a threshold. I've been using a local model and watching it do killer stuff, and also shit out useless things; all in real time, requiring active steering.
It's the uncanny valley of AI. It's still not quite good enough yet that you can trust it blindly on a big codebase, so you still have to read and understand everything - which is often harder than just writing it up yourself.
I doubt AI will be good enough to _scale_ many things because human language isn't sufficient to explain it (especially when not understanding what is being built). Humans invented more concise languages (math, C++) to pair with their intellectual acuity. [1]
In one extreme, we have art, which cannot be accurately reproduced or explained in any language (unless a concise language was used to create it, like math or CS art).
At the other end: 1 + 1 = 2 and PI = 3.14...
[1] It seems any language that has a binary outcome (correct/wrong) helps the AI tremendously, however when e.g. creating software, that software is not functioning in isolation. The software is an attempt to codify a fuzzy human system or need, and that information only flows in one direction: human to machine.
AI is going to struggle at building a consistent internal model of the domain into the software unless you’re able to give a structured explanation of the domain.
If you’re just giving it a set of inputs and expected outputs, it’s not going to generalise well and fail at out of sample input, unless the AI already understands the domain from its training set.
Being able to give a structured explanation of a domain (and being able to judge if the internal model of the software makes sense) is not the same as having experience in a domain.
Lots of ppl with domain experience can tell a right output from a false one, but can’t tell you why.
I agree, but only because YouTube is a wild west of trash, not because children somehow don't deserve to be entertained. I think that distinction should be made. Instead of focusing on barring children from the bad stuff, it might be worth trying to attract them to the good stuff. (I hear PBS Kids is a good app to leave your child alone with. No personal experience of my own though.)
You are contradicting yourself. If you're hoarding the data for yourself you're not going to develop something useful. Sharing the data means that it will be integrated into the big LLMs, which will be useful "for their own country".
>The facts show that just like the amount of labor is not fixed, neither is the size of the economy (fixed pie fallacy) and as more work is done, the economy grows
Your reply is a glib thought-terminating cliche strawman that doesn't address their point at all. Interesting!
Those theories are based mainly on the effect of Cuban immigration in Miami, however they lack a control so you can't really conclude anything.
Besides, yeah, if you hire people who will work for any salary, the amount of jobs will increase, but salaries will decrease, for locals as well. After some time, locals will flee sectors where the migrant workers are brought in, creating further self-inflicted "labor shortages"...requiring more migrants!
The main winners are capital owners, who, thanks to the migrant workers, can now acquire a larger part of the added value generated by workers.
They're available. Though you probably shouldn't invest to heavily in gen1 (production) sodium-ion batteries. It's looking like they'll be obsolete pretty quick.
I would agree if I could buy a AA battery that would power my toothbrush for a year. Or one that could be rechargeable with easily-available chargers reliably for a decade (and without having to drop 0.2V to achieve it...).
Any kind of consumer power technology can only ever be truly "good enough" if it never causes any inconvenience or significant cost.
Here are some rechargeable AA batteries that were specifically tested and worked in toothbrushes [1].
For almost all devices there is no good reason to care that the nominal voltage of NiMH rechargeable batteries 0.2V lower than the nominal voltage of alkaline non-rechargeable batteries. Alkaline batteries have a steeper initial discharge curve and pretty quickly drop below 1.3V.
If your device has trouble with 1.3V it is either going to almost instantly stop working if it is a high load device, stop working after using maybe 10% of the battery's capacity of it is medium load, and maybe 30% for a light load.
On the lasting a decade or more front, I'm still using 19 of the 24 1st generation Eneloops I bought sometime before March. 2 died and 3 are missing. Last time I went through and measured their capacities, about 3 years ago, they averaged 1886 mAh. They were sold as having an average 2000 mAh capacity with a minimum of 1900 mAh.
I've also got 15 4th generation Eneloops bought 2014-08. Those are also all still fine, with an average capacity of 1960 mAh.
You might wonder why I bought the 4th generation ones since the 1st were still fine. It is because they greatly improved the self-discharge. 1st generation was specced at retaining 80% charge after a year. 4th generation is specced at retaining 90/80/75/70 after 1/3/5/10 years. I've got some lower power applications where changing batteries is annoying, so I want to minimize self-discharge.
> For almost all devices there is no good reason to care that the nominal voltage of NiMH rechargeable batteries 0.2V lower than the nominal voltage of alkaline non-rechargeable batteries. Alkaline batteries have a steeper initial discharge curve and pretty quickly drop below 1.3V.
The few times I've measured the voltage of my non-rechargeable AA batteries (which, granted, was infrequently, and not recently), I haven't seen them drop below 1.3V until they've been in use a while.
And I've much more reliably observed that when I try to use rechargeables in my electric toothbrushes (Oral-B Pro Clean, the kind with separately moving round and long brush sections, which are, alas, no longer available anywhere I've been able to find), they start out very sluggish, and gradually descend to near-uselessness, while using non-rechargeables makes the toothbrush very energetic at the start, declining fairly steadily over a month or three, with it matching the level of the rechargeable at something like 2/3 of the way down.
I don't know what to say. I may not like it, you may not think it's actually intelligent, you may not think it's going to change the world - but how can you not see that this is revolutionary?
I remember before LLMs, someone on HN made a bot to program automatically by pulling the top rated answers from stackoverflow. To me agentic coding just feels like the next iteration of this.
And LLMs in general feel like an iteration on search.
The strengths and weakness of LLMs are already apparent, and in my opinion unlikely to change from here.
No. Nondeterministic output is not revolutionary. Technology forced down our throats by a few companies and executives who are licking their lips at the idea of laying off people, even if laying those people off means garbage products, is not revolutionary. Slop is not revolutionary.
Perhaps what people forget is that every great product builds on the past in a way to improve it. Buggy software and lame copywriting and kids not learning is not revolutionary. The people continuing to prioritize quality will be the revolutionary ones. Garbage is not revolutionary.
EDIT: don’t get me wrong. I still think AI is incredibly useful for a lot of tasks! But when implementing an architecturally hairy thing, I find it less stressful and equally quick to jump down to the editor level and use AI just for code completion.