But…and I’m sure I’m not alone here…that is a snowman, and what it is on is not a bicycle.
What do we think we are doing with this life?
fragmede 2 days ago [-]
Haha does it get bonus points for the extra button, or does it fail because html != SVG?
dodslaser 2 days ago [-]
Any bonus points for the color sre immediately subtracted because the "animate wheels" button leaves the wheels stationary and makes the sun rotate.
MostlyStable 2 days ago [-]
I wonder if it is actually animating the wheels as well, but just managed to match up the spin rate to the gap size.
cicko 2 days ago [-]
That depends on the perspective. If you're on the Sun, the wheels rotate around you.
Garlef 2 days ago [-]
Judging from the dotted trajectory lines, it even "thought" about giving the bike a wobble.
(But maybe that's just my interpretation based on something else going wrong in the animation)
fragmede 2 days ago [-]
Hy3 is a Scandinavian model, and is leaking that out via Norse mythology about Sol being a wheel!
postepowanieadm 2 days ago [-]
ROTFL
preek 1 days ago [-]
It actually rendered an SVG inline in the HTML page. I just tested the SVG and it renders itself just fine, including colors. So, tbh, I'd say the task has been properly achieved.
embedding-shape 1 days ago [-]
Maybe I'm just extremely nitpicky, but I'd consider that a failure, as the prompt is asking for SVG, not HTML.
Bit like asking for CSS and then getting a HTML file back with the CSS embedded, that was not what I was asking for!
prmph 22 hours ago [-]
Welcome to the ambiguities of natural language
PunchyHamster 22 hours ago [-]
...the animate wheels button makes sun start to spin
Just curious, can you share what are those hardest puzzles that even the top models can't crack? sometimes when I find the puzzle absolutely undecipherable I like to ask LLMs to solve it, and I haven't seen them fail yet.
aorloff 12 hours ago [-]
Ask your top model this question : I'm 100 feet away from the carwash, should I drive my car or walk ?
Tepix 9 hours ago [-]
You messed up the question.
baxtr 22 hours ago [-]
Good stuff!
Is there a reason you change the leaderboard graphs for the third and fourth one?
Also: would be great to have an overview page with a summary over all test, like a total score or similar.
CamperBob2 22 hours ago [-]
Would be interesting to see the 27B dense Qwen 3.6 model thrown into the mix.
Aurornis 2 days ago [-]
> Two new models are now beating LLM darling Claude in terms of token usage and by more than 50%?
Time for a reminder that OpenRouter leaderboards only show tokens sent through OpenRouter, which most Anthropic API users don’t use.
svantana 1 days ago [-]
I would think that's true for all the models on OR. The data is skewed for sure, but it's interesting none the less.
9cb14c1ec0 21 hours ago [-]
That doesn't mean it can't be used as a market signal. These 2 things can both be true at once.
TurdF3rguson 17 hours ago [-]
I'm pretty sure the popularity came from being free at some point
Hermes Agent 72B
OpenClaw 10B
OpenHands 9B
Claude Code 8B
Kilo Code 8B
killingtime74 1 days ago [-]
Are you next going to say YouTube rankings don't take into account videos that aren't on YouTube and Spotify rankings don't take into account songs that aren't on Spotify?
simonw 2 days ago [-]
OpenRouter rankings frustrate me, because they show the total number of tokens but they provide no indication of how many unique users a model has.
Which means if a surprise model tops the leaderboard one week we can never be sure if it was because a single whale user pushing billions of tokens a day switched to it, or if it represents a genuine community trend towards that model.
numlocked 22 hours ago [-]
(openrouter co-founder here)
Yeah we should do something to indicate cardinality. I can share that there can often (I'm talking generally; not related to this model in particular) be e.g. a very large app that can be pushing a lot of volume. But in almost all cases that app has a large number of end users. Hypothetically, for instance, would Cursor be consider one user, or millions?
Will think about it! Thanks for the feedback.
simonw 22 hours ago [-]
I'd consider Cursor one user because it's one entity that made an editorial decision about which model to make available to their own community.
If you treated Cursor as millions of users it might look like millions of people independently chose a new model when actually it was Cursor making the choice for them - and the thing I care most about is how many choices were made that selected a model and put it above the others.
dotancohen 12 hours ago [-]
An alternative viewpoint is that the single choice made about switching the Cursor model was done after extensive testing by a competent and experienced team. Whereas my naive self choosing a model to play with this week is far less a signal to others that the model is fit for purpose.
minimaxir 21 hours ago [-]
One idea I had was to count # of distinct API keys that have spent atleast $100 (number's flexible), which would be enough to provide guidance on if the traffic is from a single power-user.
In the Cursor case which is BYOK, that would count as distinct API keys.
martinald 22 hours ago [-]
Hi! Big fan of OpenRouter and the data you provide. It'd be awesome if you would consider providing volume of tokens per hour, mostly for my own curiosity as to quite how peaky demand is.
Thanks!
svantana 1 days ago [-]
Also, while we're pitching new features to openrouter, I'd like to see a "$ spent" chart, which would remove all these huge freebie spikes. It looks like it would be pretty much dominated by claude.
senordevnyc 2 days ago [-]
Agreed. My little solo dev SaaS app’s production pipelines push almost two billion tokens a day.
senordevnyc 1 days ago [-]
Haha, I never tire of the AI haters downvoting stuff like this.
Down with reality!!
daveguy 21 hours ago [-]
Or, everyone finally realizes that token burn is not the same as productivity. Maybe they just down voted for the questionable spending brag.
dotancohen 12 hours ago [-]
Questionable spending aside, GGP is providing information about how a specific metric may not measure what people think it measures. There is value in that comment.
senordevnyc 3 hours ago [-]
We were talking about whether these metrics are meaningful. I was just pointing out that even a tiny one-person company can burn a lot of tokens.
As to whether the token spend is questionable, the number I quoted is for my production AI pipelines, not for coding. And my customers (and profit margin) seem to think the spending is valuable.
andai 2 days ago [-]
So basically, Hy3 is the cheapest decent model on OpenRouter, unless you use DeepSeek as the provider for DeepSeek V4 Flash, in which case DeepSeek's insane caching wins out. (And Hy3 is close-ish on the benchmarks.)
0xbadcafebee 2 days ago [-]
You need to use DeepSeek API directly to gain the extra caching benefits. The DeepSeek provider on OpenRouter is only the 5th-cheapest for V4 Flash, so you have to specify DeepSeek provider when calling OpenRouter. But DeepSeek's API discounts on its models only applies if you call DeepSeek directly. So anyone using OpenRouter to call DeepSeek models is actually losing quite a bit of money.
NitpickLawyer 2 days ago [-]
> The DeepSeek provider on OpenRouter is only the 5th-cheapest for V4 Flash
You might have the default settings on your account, which limit Deepseek as a provider. If you disable that feature you see them on openrouter as well (and they serve it at the same cost as their own API).
However, I just double checked, and OpenRouter's pricing page for Flash v4 with DeepSeek provider shows a cache hit rate of $0.0028, which is the same as on DeepSeek's official API pricing page ($0.0028), so they do seem to be the same price, (assuming DeepSeek is able to pin your specific OpenRouter requests to the same DeepSeek server). OpenRouter adds 5% to that cost, but still it might be cheaper than the other providers.
Also just found out OpenRouter has a new feature "Response Caching" where they can cache identical requests and return them immediately with no billing. The entire request must be identical, though, not just a prefix, and you have to enable this feature. I don't know who would need to send multiple identical requests, but it's better than nothing?
NitpickLawyer 2 days ago [-]
Interesting, it seems we have some providers offering dsv4-flash cheaper than ds themselves. For the full model it's the other way around, all 3rd party providers are 2x+ more expensive.
0xbadcafebee 1 days ago [-]
The cheaper ones are fp4 and fp8 whereas I assume DeepSeek provider is unquantized, so that probably accounts for it. DeepSeek also doesn't necessarily have the cheapest hardware, other providers could be using it as a loss leader, etc
throwa356262 13 hours ago [-]
I belive no sane provider, antropic and openai included, serve BF16.
Side note: I suspect Antropic was experimenting with changing quant level based on server load a few months back which is what caused that major quality drop we saw then.
beacon294 2 days ago [-]
ZDR is also on by default and deepseek is not ZDR.
cicko 2 days ago [-]
How is it a "mysterious" model? It's Tencent's Hy3?
theanonymousone 2 days ago [-]
My question as well. Isn't Tencent a very well-known company? Maybe the mystery is in the model itself?
0xbadcafebee 2 days ago [-]
> it makes sense that a cheaper model would prevail, but only if it offered similar quality
You're trying to think logically, which has no place in an AI discussion. :) People just jump to whatever the latest model is. Plenty of people also prefer price to "quality" (which is very subjective). It's new, it's cheap, so people use it. It's likely people will stop using it when something else is cheaper and/or newer.
olmo23 1 days ago [-]
Since my employer pays for it, I just select the latest and greatest.
alecco 2 days ago [-]
PSA: Don't use OpenRouter for DeepSeek V4 as it messes up you caching. Use DeepSeek API directly and you'll get 2x to 3x more cached tokens.
numlocked 22 hours ago [-]
Can you share more? I'm with OpenRouter and we would love to address this! We don't see this in our own testing, I don't believe -- but will share this feedback and dig in.
alecco 2 hours ago [-]
Just try. In a case last week it was ~3x and I tried multiple providers: deepseek, gmicloud/fp8, novita/fp8, and another one I can't remember. It was a large job where at least 2/3rds of the start of the prompts was exactly the same (literally a static string).
Then I read somewhere (I think X) that OpenRouter adds stuff and breaks caching (telemetry? headers? can't remember). So I stopped the job, switched to actual DeepSeek provider, and voilá, caching 3x more tokens per request (on average).
bwfan123 4 hours ago [-]
Here is some data from my experience using both deepseek v4 flash directly, and deepseek v4 flash via openrouter.
Directly:
135M input tokens - $0.57 (134M cached)
Via OpenRouter
6M tokens - $0.81 (caching stats & inp/out not reported)
Caching is a huge win with using deepseek directly.
phainopepla2 2 hours ago [-]
I am experiencing this using Opencode. Caching works fine via Deepseek API but not so good via Openrouter
jaggs 1 hours ago [-]
Yes, I definitely noticed a problem with openrouter and deepseek v4 pro. It's much more expensive.
SV_BubbleTime 17 hours ago [-]
When you say Deepseek API, you mean servers in China? Or is it a copy of the model operated and run by OpenRouter?
sheepscreek 19 hours ago [-]
FYI - DeepSeek has NOT announced its own coding platform. That app is an independent project. It says so in the footer as well:
“Independent open-source project · not affiliated with DeepSeek”
Since there’s only one inference provider it could be a recycling/ad experiment. The similar usage between trial and paid periods would be explained by this as well.
thot_experiment 2 days ago [-]
Tried this extensively in OpenCode, never used it once since Gemma 4 came out, got into thought loops and did stupid edits I didn't ask for more often than the local 31b model. One of the worst "frontier" models I've ever tried.
BoorishBears 6 hours ago [-]
This article got me messing with it, and I'm loving it as a post-training target.
Training on ~1B tokens on 8xB300 and the first checkpoint halfway in learned really well. Tencent might be struggling with agentic work, but the base knowledge is there.
segmondy 1 days ago [-]
High token usage cuz it's free doesn't count
minimaxir 21 hours ago [-]
The post goes into that issue. Throughly.
The numbers at the beginning of the post are weekly aggregate values well after the endpoint was paid-only.
For the life of me I will never understand the thought process that leads you to say "we don't really know who developed this LLM but I'm going to feed all of my business's data to it"
Right but Tencent is a massive half-state-controlled holding company so that's not really helpful.
throawayonthe 1 days ago [-]
but we know who they are? how is this relevant
minraws 1 days ago [-]
OpenAI & Anthropic are deeply in bed with US govt, and they need US govt approval before model releases, and all US Companies under various acts need to share data with the govt.
I mean sure there are investors and a little more open-ness, but with the example of Mythos we don't even know if public will get access to the "good" stuff because it's too dangerous.
If your only opinion on trusting these companies more than one based in China is, they are Chinese then good luck, all the best.
estearum 1 days ago [-]
The difference is "the various acts" in the US are things that are largely very hard to do, extremely limited in scope, and companies who dispute the government's propriety can (and do) go to court to fight it.
Sure "China bad, US good" is naive, but certainly not more naive than suggesting that companies and individuals have similar rights and protections as each other.
> and they need US govt approval before model releases
This is just not true and it would be a gigantic legal battle to make it true against the model companies' wishes, which is indicative of your entire misunderstanding here.
adrian_b 1 days ago [-]
There was recently some announcement from the US govt itself (after the Mythos announcement) that they were pondering about allowing model releases from now on only after approving them.
So it may not be strictly true for the moment, but it is certainly something that the current US govt can mandate at any time.
estearum 1 days ago [-]
The US government just saying they were pondering something is:
1) Far from them actually trying to do it
2) Very, very far from them actually doing it successfully
The US government absolutely cannot "just tell" private entities what products they're allowed to create and sell, and the fact that LLMs are arguably a form of expression will make these particular products extremely hard to regulate – especially as a broad "government checkpoint" on incremental product updates.
In China, it really is as simple as the government deciding that it doesn't like your products and ta-da, you can no longer sell them.
It's beyond naive to act like these are similar in any meaningful sense.
Danox 1 days ago [-]
Nonsense, the genie is out of the bottle worldwide and it isn’t going back in, and due to the activity of the current US government America’s standing, is declining most countries going into the future are going to hedge against the United States and whatever it says the good old days (goodwill/the small benefit of the doubt) are gone.
The AI oligarchs have no loyalty and when it comes to making money and they will drop the king at their first opportunity and the king in return will do the same.
bandrami 1 days ago [-]
Well, I mean, just as a legal question I'm not allowed to use Chinese software at work, so yeah that's kind of definitive for me
nl 1 days ago [-]
> and they need US govt approval before model releases
This isn't the case (yet).
irthomasthomas 1 days ago [-]
It is for models trained with 10^26 flops. Anthropic confirmed Mythos was less than this. You could estimate the upper bound on model size from this.
nl 1 days ago [-]
That's the Biden executive order. It's notify only - the company must tell the government but the government doesn't approve or allow the release.
irthomasthomas 1 days ago [-]
Ah yeah that sounds right.
est 2 days ago [-]
> I'm going to feed all of my business's data to it
Your business data is probably worthless, even considered harmful for the pretrain corpus.
Your interactions and decision making process are most valuable parts of the whole business.
bandrami 2 days ago [-]
I assure you my business's data is not remotely worthless which is why there are pretty strict laws and regulations about what we can do with it
TZubiri 2 days ago [-]
>Your business data is probably worthless
please tell me you are not in charge of the data of any business I'm a client of
est 2 days ago [-]
to clarify, probably worthless to AI vendors, but might be useful for third-parties.
TZubiri 2 days ago [-]
Third parties that can be clients of the AI vendor...
selcuka 1 days ago [-]
If it's worthless to AI vendors, they won't include it in the training corpus, so third parties won't have access to it.
estearum 1 days ago [-]
They're alluding to something more like espionage of just selling the interesting stuff you put in the text box.
TZubiri 1 days ago [-]
Wow I thought this was quite obvious, apparently not, so I'll explain.
Llm provider sells usage of their model.
You use it to write code.
Other clients use it to write code as well.
If the llm provider trains with user data, then the usage benefits other users.
If you pay the company to generate code,then by definition it is useful, and highly likely that other customers care about it.
Replace writing code with anything, a lawyer, a psychologist, a confessional. The IO is inherently useful to users of the same category.
That is to say nothing of adversarial use, that is, being useful because a counterparty might find it useful, so an attacker might find common code patterns, a lawyer might see what the opposition might be advised, a boy might see what a girl asks or gets advised, etc..
If this sounds too complex to you, just think of training on data as exfiltration with added steps, because that's what it is
estearum 1 days ago [-]
Oh well this is a bad argument. I made a mistake by assuming you made a good argument instead.
bandrami 1 days ago [-]
The worry is direct exfiltration, not training
TZubiri 1 days ago [-]
But it isn't worthless because the user is paying for that, and third parties are paying for that as well. Unless the input output is completely different, which it's not because you are human, and I bet you have a profession which other humans have, and many other qualities which you share with other humans.
In any case, relying on the chance that the LLM inference won't train on your data because of it's presumably low value is as good a strategy as crossing your fingers or venerating the god of rain. You should be relying on contractual clauses at least when including professional and client data.
elpocko 1 days ago [-]
Could be! Let's check. I just need your name and address, your SSN, a list of businesses you are a client of, and a DNA sample.
kirtivr 2 days ago [-]
You don't need to know who developed the LLM - whether it was Google or OpenAI.
What you need to know is who is the provider for the LLM, and whether their endpoints are zero data retention enabled and opted out of training. OpenRouter gives you an easy way to control this.
lmf4lol 2 days ago [-]
This is not entirely true and ignoring a couple of potential attack vectors like Data Poisoning: https://arxiv.org/abs/2408.12798
Its of course highly dependant on the use case and the environment, but simply saying that the only important part is to know where the data goes is too simple.
koiueo 2 days ago [-]
How can openrouter control what LLM provider does with your data on their side?
kirtivr 2 days ago [-]
OpenRouter and the provider sign a contract clearly specifying how input data is to be handled.
It's the same way we trust OpenAI to not train on our data if we've opted out although there is no control on whether they can retain the data indefinitely.
lmf4lol 2 days ago [-]
I really dont want to be cynic but those guys gave a flying f””” about copyright while scraping the whole internet. How can I ever trust them to respect the oot-out setting. I cant. Thieves be thieves.
And even if they dont train on the data. Who guarantees us, they dont let another AI model analyse all the data, exfiltrating all kinds of intelligence and using it? I only can imagine what OpenAI and Anthropic know….
astrange 2 days ago [-]
Scraping the internet isn't a copyright violation. Using it for LLM training is much more transformative than Google and Internet Archive, which are legal.
jazzyjackson 1 days ago [-]
Your right, scraping is legally protected. It's reproducing verbatim text that's a violation, which is why LLMs still clumsily refuse to produce song lyrics. They are capable of copyright violations and have to be 'aligned' not to get their providers sued.
estearum 21 hours ago [-]
Verbatim reproduction is neither necessary nor sufficient to create a copyright violation.
"Copyright violation" is what we call the set of things that destroy the incentive for people to create original work by unduly benefitting from someone else's original work.
alfiedotwtf 1 days ago [-]
To be honest, this is the first time someone has spelt it out in a nicely succinct paragraph.
And just like that, I totally agree with you
estearum 1 days ago [-]
Except it ignores the entire premise of copyright which is to protect incentives to create original work, which Google does not destroy and which LLMs (very loudly and proudly) try to do.
There are several components of the Fair Use test, "transformation" is just one of them. The most important dimension is the effect on the market, i.e. the effect on incentives.
You probably shouldn't base your legal analysis on pithy internet comments regardless of how succinct or agreeable they are to you.
koiueo 1 days ago [-]
Contracts means shit if they are not enforceable.
Ask yourself
1. How would you know the provider has violated the contract?
2. How could you prove it?
3. Why would OpenRouter take your side in this (unlike your example with OpenAI, you're not a signing party)?
4. How would OpenRouter enforce the contract after all three above are somehow resolved in your favor?
IANAL, but IMO it's all a legal theater.
EDIT: formatting
ddalex 2 days ago [-]
what can it do ? it's just a big set of numbers, if you trust the host that's good enough
what266262 2 days ago [-]
If you are ok with everything being fed into it being stored forever I guess it’s no problem. I don’t see how you trust them if you don’t know them.
Dylan16807 2 days ago [-]
Who is "them" here? The developers and the hosts are not the same.
bandrami 2 days ago [-]
(And either one is a threat vector)
ddalex 1 days ago [-]
where would it be stored ? it's just a big set of numbers.
Mashimo 2 days ago [-]
If you Code open source projects anyway, might give it a spin.
st3fan 1 days ago [-]
How do you “feed data into a model” ? Use the correct terminology and concepts please. It is important.
(Transcript: https://gist.github.com/simonw/c2a0d8ecd3056a2681319eae8fc3f...)
What do we think we are doing with this life?
(But maybe that's just my interpretation based on something else going wrong in the animation)
Bit like asking for CSS and then getting a HTML file back with the CSS embedded, that was not what I was asking for!
https://github.com/lechmazur/buyout_game 10th out 36.
https://github.com/lechmazur/pact/ 14th out 25.
https://github.com/lechmazur/nyt-connections/ 60th out 81.
https://github.com/lechmazur/debate 16th out of 29.
Just curious, can you share what are those hardest puzzles that even the top models can't crack? sometimes when I find the puzzle absolutely undecipherable I like to ask LLMs to solve it, and I haven't seen them fail yet.
Is there a reason you change the leaderboard graphs for the third and fourth one?
Also: would be great to have an overview page with a summary over all test, like a total score or similar.
Time for a reminder that OpenRouter leaderboards only show tokens sent through OpenRouter, which most Anthropic API users don’t use.
The list of apps using Hy3 Preview shows Hermes Agent causing 65% usage over the last 3 weeks https://openrouter.ai/tencent/hy3-preview/apps
Which means if a surprise model tops the leaderboard one week we can never be sure if it was because a single whale user pushing billions of tokens a day switched to it, or if it represents a genuine community trend towards that model.
Yeah we should do something to indicate cardinality. I can share that there can often (I'm talking generally; not related to this model in particular) be e.g. a very large app that can be pushing a lot of volume. But in almost all cases that app has a large number of end users. Hypothetically, for instance, would Cursor be consider one user, or millions?
Will think about it! Thanks for the feedback.
If you treated Cursor as millions of users it might look like millions of people independently chose a new model when actually it was Cursor making the choice for them - and the thing I care most about is how many choices were made that selected a model and put it above the others.
In the Cursor case which is BYOK, that would count as distinct API keys.
Thanks!
Down with reality!!
As to whether the token spend is questionable, the number I quoted is for my production AI pipelines, not for coding. And my customers (and profit margin) seem to think the spending is valuable.
You might have the default settings on your account, which limit Deepseek as a provider. If you disable that feature you see them on openrouter as well (and they serve it at the same cost as their own API).
However, I just double checked, and OpenRouter's pricing page for Flash v4 with DeepSeek provider shows a cache hit rate of $0.0028, which is the same as on DeepSeek's official API pricing page ($0.0028), so they do seem to be the same price, (assuming DeepSeek is able to pin your specific OpenRouter requests to the same DeepSeek server). OpenRouter adds 5% to that cost, but still it might be cheaper than the other providers.
Also just found out OpenRouter has a new feature "Response Caching" where they can cache identical requests and return them immediately with no billing. The entire request must be identical, though, not just a prefix, and you have to enable this feature. I don't know who would need to send multiple identical requests, but it's better than nothing?
Side note: I suspect Antropic was experimenting with changing quant level based on server load a few months back which is what caused that major quality drop we saw then.
You're trying to think logically, which has no place in an AI discussion. :) People just jump to whatever the latest model is. Plenty of people also prefer price to "quality" (which is very subjective). It's new, it's cheap, so people use it. It's likely people will stop using it when something else is cheaper and/or newer.
Then I read somewhere (I think X) that OpenRouter adds stuff and breaks caching (telemetry? headers? can't remember). So I stopped the job, switched to actual DeepSeek provider, and voilá, caching 3x more tokens per request (on average).
Directly: 135M input tokens - $0.57 (134M cached)
Via OpenRouter 6M tokens - $0.81 (caching stats & inp/out not reported)
Caching is a huge win with using deepseek directly.
“Independent open-source project · not affiliated with DeepSeek”
Training on ~1B tokens on 8xB300 and the first checkpoint halfway in learned really well. Tencent might be struggling with agentic work, but the base knowledge is there.
The numbers at the beginning of the post are weekly aggregate values well after the endpoint was paid-only.
> Hy3 preview is no longer available as a free model. It has transitioned to a paid model. Continue using it here: https://openrouter.ai/tencent/hy3-preview
The Kilo Code may have free traffic but if you check the numbers is still inconsequential relative to the trillions of tokens through OpenRouter.
https://www.mdshare.online/s/uend0pj3og_A_rgcxzINf
https://hy.tencent.com/research/hy3
I mean sure there are investors and a little more open-ness, but with the example of Mythos we don't even know if public will get access to the "good" stuff because it's too dangerous.
If your only opinion on trusting these companies more than one based in China is, they are Chinese then good luck, all the best.
Sure "China bad, US good" is naive, but certainly not more naive than suggesting that companies and individuals have similar rights and protections as each other.
> and they need US govt approval before model releases
This is just not true and it would be a gigantic legal battle to make it true against the model companies' wishes, which is indicative of your entire misunderstanding here.
So it may not be strictly true for the moment, but it is certainly something that the current US govt can mandate at any time.
1) Far from them actually trying to do it
2) Very, very far from them actually doing it successfully
The US government absolutely cannot "just tell" private entities what products they're allowed to create and sell, and the fact that LLMs are arguably a form of expression will make these particular products extremely hard to regulate – especially as a broad "government checkpoint" on incremental product updates.
In China, it really is as simple as the government deciding that it doesn't like your products and ta-da, you can no longer sell them.
It's beyond naive to act like these are similar in any meaningful sense.
The AI oligarchs have no loyalty and when it comes to making money and they will drop the king at their first opportunity and the king in return will do the same.
This isn't the case (yet).
Your business data is probably worthless, even considered harmful for the pretrain corpus.
Your interactions and decision making process are most valuable parts of the whole business.
please tell me you are not in charge of the data of any business I'm a client of
Llm provider sells usage of their model. You use it to write code. Other clients use it to write code as well. If the llm provider trains with user data, then the usage benefits other users. If you pay the company to generate code,then by definition it is useful, and highly likely that other customers care about it.
Replace writing code with anything, a lawyer, a psychologist, a confessional. The IO is inherently useful to users of the same category.
That is to say nothing of adversarial use, that is, being useful because a counterparty might find it useful, so an attacker might find common code patterns, a lawyer might see what the opposition might be advised, a boy might see what a girl asks or gets advised, etc..
If this sounds too complex to you, just think of training on data as exfiltration with added steps, because that's what it is
In any case, relying on the chance that the LLM inference won't train on your data because of it's presumably low value is as good a strategy as crossing your fingers or venerating the god of rain. You should be relying on contractual clauses at least when including professional and client data.
What you need to know is who is the provider for the LLM, and whether their endpoints are zero data retention enabled and opted out of training. OpenRouter gives you an easy way to control this.
Its of course highly dependant on the use case and the environment, but simply saying that the only important part is to know where the data goes is too simple.
It's the same way we trust OpenAI to not train on our data if we've opted out although there is no control on whether they can retain the data indefinitely.
And even if they dont train on the data. Who guarantees us, they dont let another AI model analyse all the data, exfiltrating all kinds of intelligence and using it? I only can imagine what OpenAI and Anthropic know….
"Copyright violation" is what we call the set of things that destroy the incentive for people to create original work by unduly benefitting from someone else's original work.
And just like that, I totally agree with you
There are several components of the Fair Use test, "transformation" is just one of them. The most important dimension is the effect on the market, i.e. the effect on incentives.
You probably shouldn't base your legal analysis on pithy internet comments regardless of how succinct or agreeable they are to you.
Ask yourself
1. How would you know the provider has violated the contract?
2. How could you prove it?
3. Why would OpenRouter take your side in this (unlike your example with OpenAI, you're not a signing party)?
4. How would OpenRouter enforce the contract after all three above are somehow resolved in your favor?
IANAL, but IMO it's all a legal theater.
EDIT: formatting