They do heavily use AI, but you haven’t refuted their point that if inventory is in SQL, storing reservation in a second storage system increases complexity.
I mean I'm all for everything collapsing into sql. SQL all the things. Not really against it, I'd just rather not-AI write the challenges they were up against. It seems like these are all very behind-the-scenes scaling issues they faced, so it'd be cool to hear from them. Redis has great qualities, I don't use it often but I've also for years now understood why Redis was put in front of these use cases to handle them. Complexity be damned generally you're trying to enforce a first come first served or some level of idempotent behavior, so if sqls doing that now then hell yeah. It's just super off-putting to try and upend an important design pattern with an AI written article.
Yes, this is a great point and it’s even one that the Cerebras CEO spruiks for his own chips. Talaas has gone furthest on the spectrum here, so seems to have the most potential to evolve the use cases.
Taalas is developing this, but not for Frontier class models. I hope that if we can least get the easy 80% of work done on that sort of hardware, we can greatly reduce the demand for GPUs, HBM and energy to some extent.
There is an amount of brute forcing that becomes possible at those speeds that I think could even take us beyond 80%. If we could have Qwen3.6-27B running at 15k t/s, run 100 attempts concurrently, select top-K solutions and synthesize a final result from them.
There was a paper a while back that showed top-K selection like that with tiny models was able to reliably solve some 1M-step Tower of Hanoi when no frontier model could. Very big level up in capability just from horizontally scaling compute.
To extend this metaphor or the "million monkeys typing at a million keyboards", it occurs to me that if you can just slightly increase the intelligence of each sub-agent by a tiny amount, the overall end product can shoot up in quality tremendously.
If I can make my small fast AI model just a tiny, tiny bit more capable, and still run 100 of them or 1000 and run an evaluation model on top of that, the overall system capability will scale quickly with tiny increases in base model intelligence.
But some (Meta, Anthropic) suggested that optimizing and extending the "<think>" process can produce extra value. (I do not know if that requires an improved underlying architecture - frontier models architectures are sometimes not public.)
They've unfortunately been radio silent since their release, and aren't active on socials at all. I've tried to contact them for updates / api usage, but haven't heard anything back.
Looking at their career page it looks like they do not care that much about PR at the moment... considering that they have a live chat with 14000t/s via Llama 3.1 8B[0] i don't really think they need to do PR either.
So i guess maybe they currently try to solve a very hard problem with a small focused group before scaling or they are dysfunctional.
Also Llama 3.1 8B is a dense model AFAIK and they are fast by nature.
As there are not a lot of dense models these days i could imagine that they try to optimise for MOE models.
They’re not using ciphertext in inference. They are encrypting agent responses on their servers if it’s going to a subagent on the client. The subagent will send it back to their servers for inference.
Only their servers have the keys, so they can decrypt when running inference.
Yeah, if user -> org tenancy is stored in the same database without any similar defence in depth then a fresh API key after updating org would work around this.
Would be a interesting topic for them to cover.
I think the same HMAC(pepper, user, org) as a validation column would work.
Better yet, encryption with AAD on any tenancy data if you have a TPM available.
The OPs point wasn’t that OpenAis financial situation is comparable to Apples. It was that the likely cost of litigation is a drop in the ocean for OpenAi too despite their comparative lack of cash to burn.
Legal disputes like this cost in the hundreds of millions over many years, so well below 1% OpenAis last single funding round in single year.
If they got a tiny benefit from this (very gross) behaviour it may be finically well worth while.
OpenAI may very well go under IMO, but this will barely be a straw on the camels back.
That’s what they do, but the TPM pepper is also needed for HMACing in their threat model.
Otherwise the attacker just adds the victim’s user id to their hashing process too.
Zero data retention was an enterprise agreement that Anthropic and Amazon agreed with customers and delivered on.
There’s no way AWS would trade in their reputation with enterprises just to soak up some slop.
Also broadly available to us plebs via openrouter and similar. Claude is available on there under ZDR terms via the Google Vertex and Amazon Bedrock providers.
This is why I prefer open source software. I can modify it
One person can use librsync to create a Dropbox company. Another person can use librsync for noncommercial purposes, e.g., to transfer and sync their own files
I mean... I, for my own needs, which are rather simple, can replace Dropbox with rsync. That's one thing. But yes, it's an entirely different thing to consider you don't need, or worse, could own, such business, on the simple premise you don't need it at your own level. That would be madness to mistake one for the other.
The data science is where the real value comes in. 10x flags changed this release - which one caused the improved CTR? Booleans as a service need to address this, and there are benefits to having your boolean service live next to your other services
At least Anthropic claims that they are profitable on a per model basis. But since both revenue and training costs are growing exponentially, and they need to pay for model N training today, and only get revenue for model N-1 today, the offset makes it look worse than it is.
Obviously that doesn’t help them turn a profit, until they can stop growing training costs exponentially.
So it’s really a race to see whether growth in revenue or training costs decelerates first.
