Apple will release M7 MacBook Pros / Mac Minis next year, and they will be able to run free LLMs locally at native speed. All software developer notebooks will be replaced to run local models, saving a lot by cancelling Claude Code subscriptions. Developers win. Apple stocks will be rocketing. Everything else will go down. You're welcome.

You'd need the 256 gb memory model which will be expensive because apple has trouble getting capacity (got turned down by cxmt). And even then you can only run a 2 bit quant which is noticeably worse than 8 bit

Local vs remote compute is a constant thread in tech history - mainframes and desktops then local and cloud compute (think Google Photos bs Apple photos - one indexes on device the other indexes in cloud). Now we have the next chapter local vs cloud LLM models.

There will always be a market for frontier labs in the cloud based models - these models will always be able to be bigger, and that will likely translate to doing things local models can’t.

Logically also we’ll likely get to a point where RAM drops in price as production ramps up, and local LLM is both capable and cost effective. This feels like it is coming for Siri / Gemini / Alexa personal assistant type use cases.

So I think the local LLM will become a thing in laptops and phones in a year or two, offering PA type use cases. Professional LLM services will likely remain at the frontier (and in the cloud) for the foreseeable.

That’s not how that works. The hosted models don’t stay still in size and capability while Apple advances. Both will advance their frontier and there will still be a gap and developers will still prefer the stronger option.

They will cost an insane amount as well. Maybe less than subscriptions or tokens. But running massive models on laptops with batteries and poor cooling doesn’t make much sense.

Until hiding PII from the cloud LLM is a resolved issue, running local LLMs will remain a necessity.

There are workplaces that refuse to use LLMs because they fear the devs will expose sensitive data without care.

If you're a company with a considerable API bill, buying a few Blackwells and a rack could make a lot of economic sense, while still giving your security team full control of the infrastructure

> will cost an insane amount as well

We will get to a point where prosumer laptops that etch SoTA LLMs in removable silicon will be as expensive as cars.

I don't know why you'd want to burden your laptop with a large model. But I can totally see a new "developer workstation" product that's just a semi-large box that's optimized for running frontier open weights models for one to few users.

> run free LLMs locally at native speed

This reads like a hallucination. What does native speed even mean?

for example, models running at like 100-150 tokens/second (or faster!) vs 15 t/s

(fable/sol are ~60 t/s, and OpenAI just announced their Cerebras partnership(?) for "ultrafast" mode of 750 t/s)

models aren't able to run that fast right now on our consumer/prosumer hardware. M5 Max for example has a memory bandwidth of 600 GB/s. a 5090 has 3x that, so running the same model on a 5090 is that much faster (provided the model is within 30GB).

running a bigger model on an M5 Ultra is still much slower than running it on a Blackwell chip with sufficient vram, CUDA being a major difference. if apple can bridge this gap, interesting things will happen... and just imagine if M7 Ultra has comparable speeds to Blackwell (or even Rubin)!

Meanwhile the GB300 used by hosted llms:

GPU Memory Bandwidth: 7.1 TB/s Interconnect Bandwidth: 900 GB/s bidirectional

https://pi3g.com/nvidia-gb300-specifications-including-memor...

If you think M7 will hit even 15% of these speeds you're very optimistic.

A hosted instance serves multiple customers at a time. A local model only one.

How many though? At 1m context you quickly fill a full gb300's 280gb of memory

nah it won't fit that

There should be some kind of moratorium on new accounts. HN's always had waves of newcomers, but their impact was always limited. The wave passes and people either get filtered out or adapt. That doesn't seem to be happening anymore, since bots can churn out endless gibberish.

He did answer you though. Native is x10 the non-native speed. 50/50 that's not a bot; though it could be a meat-proxy

> There should be some kind of moratorium on new accounts.

OC was registered in 2016 though? What do new accounts have to do with this?

I am talking about my general impression not this particular occurence. I have a suspicion that someone/some entity is buying old accounts to bypass the new accounts penalty. I even created (https://chromewebstore.google.com/detail/hn-users-filter/ine...) to filter these accounts/comments (disclaimer: vibe-coded)

Just because someone has a take different to yours doesn't mean they are a bot or used in some mass manipulation effort.

What benefit exactly is achieved by selling a 2016 account (decade old) to then post about Apple potentially releasing high memory Macbooks next year?

This sort of elitism is the same nonsense that killed StackOverflow.

I assume they mean same t/sec as a SOTA cloud model

Sure buddy, all you'll end up with is a $10k machine that run gimped models at like 30tok/s for about 5m before the fan kicks in and it starts to sound like a turboprop, while offering maybe 30% of the context size of hosted models.

Every single MacBook built in the past half-decade already has an LLM built into the latest version of their OS.

But there's a significant difference in hardware required between running a 3B parameter model and a 700B-1T+ parameter model.

The RAM shortage situation won’t be sorted out within the next year.

Bold to believe it will be sorted at all

If the margins are there it will get sorted.

I'm still waiting for Linux to topple Windows

The processors running the Linux kernel outnumber those running Windows 4:1.

> Apple will release M7 MacBook Pros / Mac Minis next year

The latest on Apple is TSMC is stuck on the next iPhone due to lack of RAM. Good luck getting any Macs. Memory shortage is getting worse.

I'll give you credit for at least offering a specific, somewhat unique take. But this is a pretty dumb take lol

Both the example libraries compared (tokenizers and tiktoken) are Rust-based with Python bindings. There's just a few levers in Rust that can speed it up even more particularly with LLM assistance as the AI Use Discloure here notes:

> Final profiling stages and the last ~4x worth of performance from eliminating branching and improving the pretoken cache hierarchy

1 year ago everyone would have called you insane for suggesting this. Now we all shrug and say yeah maybe we can do this and it’s actually a good idea?

My choice is https://www.skeleton.dev/

Why? There's a thousand component libraries, what makes this one stand out?

It uses Zag js for its primitives which is framework agonistic as opposed to Radix or Base UI. Shadcn is only for React. If you use Solid.js for example, Skeleton is an option for your design system.

There are ports of shadcn for svelte, vue and solid.

There’s also Basecoat, which is shadcn for vanilla JS.

https://basecoatui.com/

How about following the test-driven approach properly? Asking Claude Code to write tests first and implement the solution after? Research -> Test Plan -> Write Tests -> Implementation Plan -> Write Implementation

There was an ad built on this conspiracy. :) https://youtu.be/HynsTvRVLiI?si=Qhmm8LSztsXeWyzp

This is my favourite comment on this topic:

"In the 90s I was on the Disappearing Team for the McGillicuddy Serious Party. When we were in power my task was to go around the world erasing New Zealand from all the maps, since if no-one knew we were here, we wouldn't have to spend anything on defense or build annoying tourist traps" (c) RalphNZ

I guess it is still useless in Ruby or Ruby on Rails. Standard "find the method declaration" or "used here" do not work in Ruby on Rails. Still, huge companies maintain that Ruby on Rails mess, where you cannot properly investigate, so you just guess and use the search and find option. Those codebases won't be replaced for a while, but good luck working on them. Such a headache!

Man am I glad I don't have to work with Ruby anymore

How come nobody came up with an LSP that can perform this, all this time ?

I've developed a small tool[1] that has helped me for the same problem, but in Python. Basically just uses simple parsers to attempt to find a definition wherever is sensible. Adding a Ruby module should not be too difficult, but it would probably be trickier than Python to get some good enough results

[1] https://github.com/federicotdn/irk

Because it's an unsolvable problem without static type annotations and as far as I'm aware Ruby doesn't have a good solution for those yet (or if they do nobody uses it).

Sorbet is a decent one. I don't think it ever solved "jump to definition" though. I would just `rg def (self\.)?function_name` or I eventually developed a vibe for where things were, which is sort of the Ruby excuse for the ungreppability of everything. Sorbet did allow us to generate front end types in Elm and also allowed for type safe Haskell FFI. Past tense because it's an old job; as far as I know it's still happening.

Someone needs to make TypeScript for Ruby.

https://shopify.github.io/ruby-lsp/rails-add-on#go-to-defini... ?

First of all, ruby-lsp does a great job at this, and the recent Herb helps with frontend templates.

This is enough to navigate between controllers, models and libs, unless you're trying hard to be clever which you shouldn't.

Then, in Rails, things have a canonical place in the codebase, that is consistent between codebases.

This is in contrast to languages and frameworks where every codebase is setup differently, but the static typing helps find code wherever it's hidden without pain, and thus without need for cleanup and thoughtful design.

To each their own, I prefer power for me, and pain for whoever drifts from the convention.

I loved RethinkDB. I'm sad.