> If the model is nothing more than the sum of its training data and regime, then the company is responsible for its behaviour.
What stops the company from being responsible regardless? They created this entity, it's running on servers they own or rent, and (in these cases) it's acting on their instructions.
If it's also conscious, then IMO that greatly broadens their moral responsibility, because now model welfare matters. But we're talking about their responsibility for the model's actions, and I don't see how this could be weakened by model consciousness, given all of the above. As for their legal responsibility, the models don't have legal personhood, so who else but the company could be responsible?
It gets more complicated when the person who sets the model in motion (i.e. prompts it) is a third party, but in cases of internal models committing cybercrime during testing, surely the locus of responsibility is obvious.
On my read they are criticising the reporter, not the child. We definitely should be careful, in situations like this, not to be nasty to the kids involved. But I think we should still be allowed to be upset by bullshit, and strongly criticize the bullshitters, even when they are bullshitting about a child.
> Please tell me that this is a grossly exaggerated parody, and that the tools don’t write like this, or do so many ridiculous things
It's a pisstake, but (in the bits I read, and based on my own personal experience) the writing style is barely exaggerated, while the behaviour doesn't ring true at all.
The behavior, while slightly exaggerated, rings entirely true for me. From the other comments in this thread, it seems I am prompting poorly in a similar way to the options offered.
I am guessing you prompt differently than what is shown in the game?
I'm not the person you were replying to, but yes. Let me show you the issue.
"Why is half the site blue now? I asked you to change one button."; half the site isn't blue, why would you say that? What would "half the site" even mean? Ironically even in this satire supposed to make fun of how Claude responds to prompts, it is smart enough to ignore that. A better prompt would be "Why are many parts of the site, such as the Cancel button, now also blue?", though in this case it doesn't matter as the reply would be roughly the same.
Next, Claude helpfully tells us we're dealing with a monstrosity of a codebase out of hell: "Seven components consume the token directly; another eleven reach it through aliases; three use it only in hover/focus state; and two appear to be accidental cross-role consumers" is such an insane codebase that the initial request was basically impossible to carry out.
Claude made that perfectly clear, yet the next prompt choice just ignores that fact. "Revert everything except Add to Cart. That is the whole task." It just explained why this is impossible. It had only made one change, so there is no "revert everything except X"; there's only a single change to revert, which would mean Add to Cart too would no longer be blue. The human just chose to ignore that. The other option is "I do not care about the token architecture. I do not care why it happened. Put everything back the way it was and leave ONLY the Add to cart button blue. Please.". This has the same issue; "Put everything back and _leave_ only the button blue" is an impossible ask, which it just explained to you. So if you really don't want to deal with cleaning up some of the mess, instead only adding on to it but getting your blue button, then you'd want to say "Revert the change you made, then make a new change that scopes the new blue color to only the Cancel button".
Ignoring the information it gives you, or telling it things that are incorrect, of course will lead to bad results. It's impossible not to.
The behavior is congruent with my experience in abstract, in that all models will regularly do things you didn't ask for, will regularly go "above and beyond" by their training I expect, will regularly make changes that are entirely orthogonal to the change you asked for.
I've worked with Opus and Sonnet daily, and they are pretty great at generating functions and modules and components that have clear boundaries of concern, but I've recently been working some research tasks into our infrastructure and code and it seems impossible to coerce Sonnet into making only specific changes to a document you are working on.
It also blatantly ignores instructions as a rule. "Don't disassemble java class files, just ask me to pull in the source code" worked less than half the time. The Intellij Copilot plugin just doesn't use the AGENTS.md and similar files, and there doesn't seem to be any meaningful activity in the bug reports of same. "Don't modify code unless I tell you to" had bad adherence as well.
It also will read documentation and inform you that it says the opposite. This problem happened to me across models, across model updates, across months of real time. There's a specific example that I will not mention to avoid having it be trained on specifically. A distinct but similar problem is that it will take bad documentation and just pretend it has a good understanding. Claude gave me absurdly wrong descriptions for Splunk alert settings with absolute confidence.
I don't think any agent can reliably figure out "I don't know"
I appreciate that he responded with (seeming) openness and detail, rather than just posting some pithy insult or whatever would have won him the twitter battle, but this part feels like serious gaslighting or, at best, self-delusion:
> Genuinely, at that moment, I was trying to care for him and do a last ditch attempt to get a chance to give them all the credits that they deserve.
The allegation he is responding to, and which he does not seem to have disputed, is the following passage from Buckmaster's statement (https://cims.nyu.edu/~tristanb/statement.pdf):
> I said that if OpenAI released its result in the way proposed I would go public with what happened. The reply was, “Why would you ruin your career?” I replied that I am an academic, and asked why he thought going public would ruin my career. The reply was, “If you don’t want me to be nice, then I don’t have to be nice.”
This is helped along by the number of people willing to cynically offer up a blurb, or even a glowing review, to something mediocre or seriously flawed that they may not even have read. I'm sure the blurbers think they're just doing someone a favour (and/or playing the game so that they will receive reciprocation), and the reviewers are probably less often crooked than simply busy/lazy, but it all goes to make it much harder for good, substantial work to stand out.
Maybe I misunderstand the analogy, but in this scenario it seems that the orphan crushing facility's donations to the orphanage are an unalloyed good. Their motives are bad, and they are bad, but their donations to the orphanage are causing more orphans to be taken care of and fewer orphans to be crushed.
Personally, my worry about anonymous donation matching would be that the anonymous (to me) funder is influencing the recipient (to whom their identity is known), and without knowing who they are I can't know the nature of this influence or whether it should cause me to reconsider my support. But I don't see any extra reason to care about anonymous donation matching in particular, as distinct from any other funding whose source is opaque to me.
Oh. Right. You just introduced the concept of non-anonymous (to the orphanage) matching funds. I didn't think of that.
More money for the orphanage means more orphans can be taken in. That's good, right? But since the extra funds are not anonymous (to the orphanage), then it can come with influence on policies.
Policies like: All orphans are now accepted up to a certain age regardless of circumstance, and all orphans now get kicked out at age 12 regardless of ongoing need.
The stated reason is that the orphanage only has so many resources, and little kids need a lot more care than 12-year-olds do. So a line must be drawn somewhere, and that line is at age 12.
It's a popular policy: After all, when given binary choice, everyone would rather see a 3-year-old be taken good care of than a 12-year-old. Think of the little children!
For the orphan crushing facility that helped to impart this policy with their very generous policy-adjustment matching funds, that's a solid win: The orphanage now acts as a livestock kennel where orphans grow in size. There's still fewer of them coming out the far end, but they're bigger than they were before and are now much easier to process in bulk.
People know about the orphan crushing facility. They know they can't do anything about it; it just looms over there like any other dark fact of life. But the streets aren't full of roaming herds of abandoned kids anymore at all, so folks logically conclude that the 12-and-out policy must be working.
I don't like that phrase either, but do you have a direct replacement? Language usage tends toward efficiency, and sometimes I want to speak generally rather than specifically, and so if my options for conveying the idea 'read books and/or listened to music and/or saw plays and/or watched tv and/or ......' are to write that out or to say 'consumed content', I'm probably going to hold my nose and go with the latter, at least once I've heard it enough times for it to feel relatively normal.
edit: actually, and this is just a tangential point, I don't think 'content' covers plays or other live experiences. It's usually screen-based, and the main possible exceptions are things that are almost interchangeable with their screen-mediated forms, like books or magazines.
On desktop, it's basically forced heavy smoothscrolling, with an attempt to replicate real motion by gradually slowing to a stop. Horribly laggy for those of us sensitive to such things.
I'm one of today's lucky 10,000, so I'm glad it was linked here (and would happily read a defence of the language from one of those previous discussions, too).
When the criticisms relate to correctness bugs, I don't think 'try it out and see how you like it' is sufficient. I might love the syntax and the design and so on, but that doesn't tell me whether I'm going to run into serious bugs some time in the future.
That reads to me more like a long-winded example of a Julia user refusing to take correctness issues seriously, and instead using an LLM to self-soothe by deflecting onto other projects:
> I think there’s also a mindset split, some people just like to have things more strict and avoid bugs by having their compiler proof everything, and others like more freedom and are fine with occasional mishaps.
> Just for the fun of it, I put claude on Python, and it also found some eye watering correctness issues (to be fair, I haven’t taken the time to verify and judge them, but it seems like that’s a similar situation for the Julia version)
I say "self-soothe" because if the intention were to better understand the correctness situation, presumably one would at least want to evaluate the output before declaring it "eye watering". And then even if the output was real, it would be better to report it to the affected Python projects instead of using it as an excuse to downplay problems in Julia.
But most of the supposed "bugs" seem like totally fine/reasonable behaviors to me, often for clearly nonsensical inputs. Seriously, `np.array([1, 'two', 3.0])`? That's not a bug, the behavior is clearly documented on numpy.org, but really no matter what Python does with that, it's not comparable to issues like `prod([Int8(100), Int8(100)]) != prod((Int8(100), Int8(100)))` from that post about Julia. Which again the linked Discourse post downplays as "freedom and occasional mishaps".
You can not be serious in suggesting these aren’t straight up Python correctness bugs. Exactly the same kind that Yuri brought up as damning evidence of Julia unseriousness, but for Python with easily 25x the user base.
Most (all?) aren't bugs by any stretch of the imagination, no. Let's go over the first 5.
1. random.choices(['a','b','c'], weights=[-1,5,1], k=10000)
Negative weight on 'a' silently shifts
Python docs say, "Weights are assumed to be non-negative and finite." Garbage in, garbage out.
2. random.choices(['a','b','c'], cum_weights=[5,2,7], k=10000)
Non-monotone cum_weights makes 'b' unselectable.
...Those weights aren't cumulative, which the docs say they should be. Again, garbage in, garbage out.
3. statistics.fmean([1,2,3], weights=[-1,1,1])
“Mean” of three values in [1,3] returns 4 — outside the convex hull.
This is just straight-up mathematically correct behavior. It preserves linearity. It fits the commonly accepted definition of weighted mean as `(w1*x1+w2*x2...)/(w1+w2...)`.
The LLM fabricated a fake/idiosyncratic definition of weighted mean in order to claim it's a bug, because it was instructed to come up with bugs.
4. json.dumps({1: 'a', '1': 'b'})
Produces invalid JSON with duplicate keys; round-trip silently drops one entry.
Again, documented behavior/GIGO. Docs say, "loads(dumps(x)) != x if x has non-string keys."
This is literally just what geturl() is supposed to do. It's the whole point. Docs say "empty parameters, queries, and fragment identifiers will be removed". The LLM is claiming that geturl()'s primary intended purpose is a bug.
So all of these "eye watering correctness issues" so far seem to be either (1) straight-up correct, or (2) doing things Python explicitly tell you not to do. Same deal with the Numpy "bugs", AFAICT, as I touched on in my previous comment.
In fact, I would venture that we all know those Python bugs are fake, but (unfortunately) the Julia ones aren't. Because the Julia bugs mentioned by Yuri were reported to the Julia bug tracker, and eventually fixed. Whereas if you really thought these are real bugs in Python, then (IMO) you should be reporting them to the Python tracker, not getting mad at me for doubting them.
Moreover, even if they were real bugs in Python (which they aren't), bugs existing in Python still wouldn't change the situation for Julia. The Discourse user who posted it still admitted that they didn't even take the time to verify them.
Surely you must realize how bad it makes Julia look, when its users fling LLM slop to attack Python in response to Julia's issues being brought up? A constructive project should instead talk about what's been done and planned to improve Julia's situation, not tell lies to drag Python down. I liked Julia when I tried it! The JIT plus multiple dispatch is so unique. But this so isn't the way.
python has plenty of bugs like these too. is this example also "LLM slop" ? I think it's frankly delusional to somehow believe that these issues are unique to Julia.
> But most of the supposed "bugs" seem like totally fine/reasonable behaviors to me
exactly. and the same is true for many of the bugs that have been presented as indictments of Julia. but when the same is said of those, the community is called "defensive." so it's a lose-lose.
There is no equivalence here. The Julia bugs were real. They were reported, accepted, and fixed. The Python bugs you linked to are fake LLM slop, see my other comment right above/below this one. [0] If anyone really believes the Python bugs are real, they should report it to Python, not use it to deflect from Julia's issues.
What's been presented as an indictment of Julia (in Yuri's own post and after) is the fact that members of the Julia community have vocally downplayed problems and played the victim when quality concerns have been raised, as I think you're doing. Do you want to convince everybody you've "won" "a lose-lose"? Or do you want to write correct programs?
I like Julia, the language and the tech. I really hope this hostile attitude towards criticism and growth fades eventually, because I'd like to be able to use and trust it at some point.
I wouldn't say this is true really. Julia does have some unique properties which cause these issues other languages just sidestep. The dynamic dispatch system is really magical when it works, but it's the source of much of the consternation you see here in this thread, and the reason it persists despite individual bugs being fixed. The problem is the "bugs" in this case aren't really as such; they're not wrong code, they are violations of silent contracts.
The whole magic of dynamic dispatch is you write Library A and Type B, and they "just work" together without having to know about one another. This is of course very powerful and so people have been very enthusiastic when wielding it.
But with great power comes great responsibility; when using libraries and types that weren't meant to work together, one of those types might violate a silent contract in the library. This would be fine if the error could be caught at compile time, but it happens in the form of numerical correctness issues, so they don't even present as actual errors.
The most obvious example of this is where Julia allows for arbitrary arrays and two things expecting different bases come into contact. This is something that's just not possible in other languages, so they're not exposed to this class of bugs.
So maybe you can harden and make explicit some of these contracts, or put up warning signs, or add some lints, and thus "fix bugs"; but they keep coming back because the dynamic dispatch system assures it due to the combinatorial explosion of interactions it incurs. I'm very interested in how Julia will solve this issue going forward.
What stops the company from being responsible regardless? They created this entity, it's running on servers they own or rent, and (in these cases) it's acting on their instructions.
If it's also conscious, then IMO that greatly broadens their moral responsibility, because now model welfare matters. But we're talking about their responsibility for the model's actions, and I don't see how this could be weakened by model consciousness, given all of the above. As for their legal responsibility, the models don't have legal personhood, so who else but the company could be responsible?
It gets more complicated when the person who sets the model in motion (i.e. prompts it) is a third party, but in cases of internal models committing cybercrime during testing, surely the locus of responsibility is obvious.
reply