Weigths directly in silicon is a bad idea with the way the space is pacing. Just look at chatjimmy.ai it is fast, but on the once good llama3.1-8b but now pretty useless.
That's why I believe the baked weights should represent a generic model that is good at general thinking and tool calling. The rest can be augmented by the OS.
