But I feel the only reason Chinese models have been able to gain such popularity is BECAUSE they are open source.
I wouldn’t trust models from China enough to use them in important work unless they were open.
Restricting any powerful Chinese model [that ranks in the global top 10] is going to make them very difficult to advocate for, and very hard for global users to adopt.
Feels like this could be the beginning of the end of the democratisation of powerful AI.
And yes, I appreciate the irony that Chinese models have been the ones enabling that democratisation!
Latest and greatest China is able to make is reserved for the use of businesses and entities in the Chinese market.
Everyone else gets access to models of more limited capabilities.
It's basically the US strategy. Obvious difference being only that the Chinese allow on prem hosting of the lesser models. So plus one to them I guess.
If we want to move China or the US on from the whole, "super intelligence models for me, more limited intelligence models for everyone else" strategy, then "everyone else" will have to come up with some models competitive with the latest and greatest from the US and China. Absent that, there's just no pressure to bring the US and China to the "share the wealth" table.
Unfortunately, "everyone else" coming up with a competitive model is not gonna happen anytime soon.
It makes it harder to extract decent margins from models at or below the quality of the open models, and so makes it harder both for new entrants and for any existing providers that are depending on cheaper models for revenue. At the same time you make people comfortable with your product.
It's the IP version of dumping products below cost if you're capital rich and want to winnow down the competitive field while "buying" customers.
A model is never neutral. A model is always also a way of disseminating an ideology, and it doesn't need to be lying for that to work.
Consider the question "what caused the French revolution?"
There's many ways to answer this while being technically correct. What gets prioritized, what gets omitted, that's where the ideology lives.
(Unless someone totally bonkers spends thousands of hours and thousands of pages answering a single question)
But once we automate that, it drastically lowers the amount of viewpoints represented.
But if there's an open model available then they might choose that instead of either of them. Even if the Chinese models aren't used they will bring down profit margin just by existing. They wouldn't have the control the hosted American models give but that wasn't even on the table. But denying America control is a worthwhile strategic goal in and of itself. And they don't even need to expend extra resources to do it, they needed to develop their own models for security reasons anyway.
So you've inspected the weights for backdoors?
If you're using a closed model, the model provider gets all your data, and can if they want to send back actively hostile tool calls and rely on your harness to catch them without any effort on their side to train that behaviour into their models. The only things you can do about that is to place careful restrictions
If you're using an open model against your own system or a third party provider, the origin of that model could have trained in "triggers" that'd cause the model to produce actively hostile tool calls as well, but they'd have no direct control and would need to train in that behaviour "blind" to how you'd actually use the model.
It doesn't mean you can be complacent about trusting the model, but it does change the likelyhood they'd be able to do any damage, not least because you'd be able to run evals against the model without any risk that the model or provider API could just randomly change on you after a while.
So it takes far more trust to run a closed model than to run an open model even on a hosted third party provider, and certainly far more than to run an open model on your own hardware.
Why do people keep calling these "open source" models? They're not that different from NVIDIA's binary blobs. Sure, they work, and you can run them and get results, but the source code, training data, and training process are largely hidden and proprietary. Publishing weights is not the same as publishing the source. They're open-weight models, not open-source models.
"Second, we encourage open source, open collaboration, and resource sharing. The Chinese government is committed to making artificial intelligence a significant driving force for promoting common prosperity and safeguarding common security. To support the construction of an open source innovation ecosystem for artificial intelligence, convenient measures such as automatic licensing will be applied to technology exports involving open source AI technologies, Chinese companies building and operating their own computing centers overseas, conducting chip fabrication overseas, utilizing overseas resources for joint research and development, and participating in the formulation of international standards, basically achieving "immediate reporting and export." Technologies that have already entered the public domain through open source or published academic papers will not be included in the export management scope, creating a favorable environment for international exchange and cooperation in artificial intelligence technology."
their official stance suggests they remain in support of open-weights
What they are saying is that they won't be changing anything.
That could of course change overnight.
There are multiple aspects to this: China is cottoning on to the meaning of soft power, and besides that being a major supplier of AI related models, services and hardware further cements their place in the world economy. They may not be able to compete directly with the USA in this domain but they are able to undercut them.
Of course there are risks as well. Models produced and released into the wild for which you have not seen the training and conditioning data could have payloads that you'd rather not have. But the same thing goes for service providers in other countries as well. You can't really trust hosted inference, not to tell you the truth in so far as is was present in the training data (censorship, not inference errors) and not to leak your information.
So they're changing some things.
“Right now, your government is doing things you think only other governments do.” -Van Halen, Right Now
My (not thorougly informed) take is that they are are taking measures to have the capability of restriction but are also adding provisions so that this capability does not add friction to the current practice of enabling export of open weight models.
So to me it's like they are keeping their options open rather than leaning on one versus the other.
What they will do remains to be seen. The document at hand is what they're saying.
I.e. less important if we follow your own assessment.
Says who?
Part of the problem is that there are too many major stories happen back-to-back nowadays, and important stories like this one never get discussed. HN only shows 30 stories on the front page, which limits discussion.
Additionally, the financial and technological equilibrium which makes OpenAI or Anthropic possible is rather precarious so they are poking on it, hoping to make it fall.
So the real question you should be asking yourself is why you thought that China supported publishing open-weight models, even though they merely didn't prohibit it.
The Chinese open-source models (which are 1-2 generations behind the frontier) cost in the low tens of millions to train. This is largely because of a lack of compute. They achieve comparable performance to frontier models via distillation.
somebody wake me up when we have "SLSA, but for model provenance".
I don't (just) want "open weight + verified distribution". I want a dang audit trail.
If/when one of these models is trusted enough to make real decisions with teeth, and if/when one of those decisions screws up badly enough to hurt somebody, I want to be able to play back the tape and pinpoint the source dataset + training method responsible for the failure. At LEAST at a coarse level, on par with something like modern RPO SLAs.
P.S. this comment is applicable to pretrained models in general. I don't care if it came from China, USA, or model weights scrawled in sharpie on a random bathroom stall. I just want to know the provenance and have enough information to run a blameless "git blame" (or equivalent) if/when things go wrong.
P.P.S. IMHO, we aren't there (yet), but OTOH this proposal ain't a bad place to start.
This was a topic of discussion at Trump and Xi's AI summit just last month. We're now starting to see the export controls coming into place.