While this problem isn't exclusive to Claude, Claude does seem to be the most prone to it in my experience. I've had very few, if any, "WTF that's exactly what I told you not to do," experiences with other models. Codex in particular seems to be excellent at direction following and not breaking rules.
There's another layer to the non-determinism of LLM agents: what are the execution params the provider is using today?
I hate the feeling that a worn path that I've grown to trust will "do the right thing" over the last few months will suddenly start doing the wrong thing simply because an engineer at Anthropic or OpenAI found a way to save N million dollars by "optimizing" thinking token usage.
Having your "workstation" with monitors floating around you in space wherever you're sitting or standing with zero cable management. Whether you're at home in the comfy chair, at a treadmill getting your steps in, or at a hotel on a work trip.
Once the resolution and UX gets good enough a lot of people would love to have their entire office setup replaced by a portable wearable with next to zero cable management. Doubly so if that opens up space in your expensive SF apartment.
That's all good in theory, but we're still a long, long away from this being the future, let alone a future that everybody wants.
I’d probably do more development in the meta quest virtual desktop environment if my primary occupation these days wasn’t writing software for the meta quest. It makes it difficult to iterate when you constantly need to flip to a mode where your monitors disappear.
> This includes not clearing/compacting the context often. Opus now has a 1M context window, and quality is good to at least 200K. So each query is burning a lot of tokens until you clear/compact.
I see this repeated by others, including coworkers. It completely ignores caching. Caching itself is complicated, but the "longer context window = more expensive" is not 100% true and you are hampering yourself if you're not taking full advantage of large context windows.
You still pay for cache hits and refreshes, but the cost is lower.
The default Claude cache expires in 5 minutes. If you take a short break to review the code, talk to someone, or do anything other than continuously interact with the session it's going to get evicted and start over.
Also anecdotally, caching has just been broken at times for me. I've had active conversations where turns less than 5 minutes apart were consuming so much quota that I doubt anything was being billed at the cache rate.
If you look at the actual cost of your Claude Code conversations, you'll see that the cost is overwhelmingly dominated by the cost of input tokens (cached). Because of how we construct persistent conversations, each cached input token incurs cost on each API request, meaning that component of cost scales with O(request count). If you graph the cost curve of a claude code session, it's very obvious that this scaling factor overwhelms the cache discount.
Here is a blog post that shows some data - https://blog.exe.dev/expensively-quadratic. And I can confirm this is true for Claude Code - I set up a MITM capture for all Claude Code requests and graphed it.
So increasing Request Count that reuses the same prefix (which is what higher compaction thresholds do) really does lead to (substantially) higher API costs.
Caching is pretty simple. If it's a prefix match, it's cacheable. Very long context windows will be much more expensive than shorter ones, even with caching, assuming you're using Claude Code or some similar harness for both. You'll get caching in both, but you'll pay more for the longer context. The cost of occasional compaction is more or less negligible compared to the massive cost of the input tokens that are getting charged repeatedly for every single request.
If you have 500k context, three turns will burn ~1.5MM tokens. If you have 250k context, three turns will burn ~750k tokens. If you have 125k context, three turns will burn ~375k tokens. Claude can at most generate 32k output tokens per turn in Claude Code (and it rarely does so), so despite the higher price of output tokens, almost all costs are dominated by input token costs. Even at cached input prices, cost scales near-linearly with context length: if you 2x your context length, you'll roughly ~2x your cost.
Now, it might be the case that longer context windows allow Claude to complete the task better — although I'd be surprised if there were many tasks requiring >200k tokens just to get the job done (that's nearly ten full copies of Shakespeare's "A Midsummer Night's Dream"). And they're definitely convenient, in the sense that you don't need to think about context management as much and worry about a sudden, unexpected autocompact wrecking things if you aren't carefully manually compacting at logical points. But they're definitely more expensive on a near-linear basis and you're paying for that convenience.
You're talking about the "cultural collapse" of both Japan and Russia as if it was common knowledge. What exactly do you mean by this? Is this your personal opinion, or a reference to some quantifiable metric?
Japan is currently one of the hottest tourist destinations in the world. First because of the strength of the dollar vs the yen, but also because of their culture.
ChatGPT and LLMs have had a significant impact on my wife's life. She's a second language speaker, and having ChatGPT available to draft and proofread professional sounding emails and text messages has drastically increased her self-confidence and ability to communicate with colleagues. I think that's amazing.
Two uses for me (as a native English speaker who writes pretty well on my own):
1. Reformatting notes or bits of information into something more formal (something I consider actually counterproductive in a way, since formal is often more verbose, but that's expected in certain contexts...)
2. Sifting through the crap of the internet to answer obscure questions. The Google replacement that has been needed.
It's helped me incredibly to proof-read a novel I wrote in Spanish and translated myself into English to make it sound more native. I review ever single suggestion an LLM provides (as I would do with a native proof-reader!).
I think this type of job suits LLMs perfectly... At the end of the day it's just a statistical NLP tool.
For second/new language users, search enabled LLMs are great for finding information. You can instruct it to search in the target language and provide the results in your native language, helping greatly when you're not even sure what the relevant search words are.
I wouldn't trust the analysis on anything important, but that gives you the source links so you can still verify yourself.
The downside to doing this is that you'll sound like an LLM. LLM-generated text is very obvious to anyone with basic reading comprehension and once detected will cause some people to summarily dismiss the sender as a bot.
This is more than acceptable if it allows you to confidently send of an email in less than a minute that would otherwise take you 30 minutes of agony to write and still not be confident about.
Also, these aren't cold calls. The recipients aren't critical about how "botty" the email sounds.
I feel like the fact that you are able to say this, and the sentiment echoed in other comments, is a pretty decent sign that the "movement" has peaked. It was just a few years ago that anybody voicing this kind of opinion was immediately shot down and buried on this very forum.
It will take a while for DEI to cool down in corporate settings, as that will always be lagging behind social sentiment in broader society.
I see it more as the simple truth. There's only so much influence you can wield working one day a week as an executive advisor. By his own admission he could have steered things better if he'd been more involved, but he didn't want to be more involved. He's got his own startup to work on.
Oh yeah I was assuming we were talking about when he was actually working there. That whole consulting gig felt like a slow quit.
> By his own admission he could have steered things better if he'd been more involved
My reading is that back when he was full time, he was busy with actual product work and coding, and he could have possibly made a difference by going political, but there would have been no guarantee and it would have taken a lot of his time.
Yup. I had the misfortune of being in Michigan, unemployed, at a particularly bad time. I applied to about 30-40 gas stations, movie theaters, fast food places... Everywhere I went I was told the same thing, "I'm required by law to give you this application form, but we're not going to hire you. Good luck." I didn't know anybody and couldn't get a job cleaning floors let alone flipping burgers.
unfortunately it does take time but eventually those not paying you, me, us all, they stop making money by sitting on it, then they have no choice but to spend
> I find multi-letter variable names extremely old fasioned
Sometimes I read things here on Hacker News that throw me so hard I leave the site for a month or two. Congratulations, this time it's your fault. Goodbye.
One-letter variables have all approximately the same physical size; this makes the “tokenization” step of reading a formula faster.
They are also less descriptive, and this makes the semantic interpretation more difficult.
Usually mathematicians read entire papers, or large excerpts, at a time. In this situation the semantic association symbols<->concepts is often made at the beginning of each section and reused for several formulas, making mathematical notation more effective.
Programmers instead often look at code in smaller fragments. They don’t have thesame level of contextual information readily available and so they often prefer to embed this information in variable names.
Add that programs are written mostly in ASCII, on a keyboard, with autocomplete, in a single typeface, and math by hand, on paper or blackboard, with much more graphic possibilities.
