Solid bird, not a great bicycle frame.
Context for the unaware: https://simonwillison.net/tags/pelican-riding-a-bicycle/
Do electric pelicans dream of touching electric grass?
That would be shocking news to me.
I'm guessing both humans and LLMs would tend to get the "vibe" from the pelican task, that they're essentially being asked to create something like a child's crayon drawing. And that "vibe" then brings with it associations with all the types of things children might normally include in a drawing.
It's just an experiment on how different models interpret a vague prompt. "Generate an SVG of a pelican riding a bicycle" is loaded with ambiguity. It's practically designed to generate 'interesting' results because the prompt is not specific.
It also happens to be an example of the least practical way to engage with an LLM. It's no more capable of reading your mind than anyone or anything else.
I argue that, in the service of AI, there is a lot of flexibility being created around the scientific method.
For the last generation of models, and for today's flash/mini models, I think there is still a not-unreasonable binary question ("is this a pelican on a bicycle?") that you can answer by just looking at the result: https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/
I've worked one an RLHF project for one of the larger model providers, and the instructions provided to the reviewers were very clear that if there was no objective correct answer, they were still required to choose the best answer, and while there were of course disagreements in the margins, groups of people do tend to converge on the big lines.
It is unreasonable to expect pelicans to ride human bikes, they have different anatomy.
Draw a pelican on a bicycle ergonomically designed for pelicans.
For reasons, I have tried to get Stable Diffusion to put parrots into spacesuits. Always ended up with the beak coming out where the visor glass should've been, either no wings at all or wings outside the suit, legs and torso just human-shaped.
ChatGPT got the helmet right, but their wings and tail (and sometimes claws) were exposed to vacuum, still very much closer to a human in either a normal or scifi space suit that happens to also be wearing a parrot head inside the space suit, and has tacked some costume wings on the outside.
Essentially, it's got the same category of wrong as fantasy art's approach to what women's armour should look like: aesthetics are great, but it would be instantly lethal if done for real.
> Generate an SVG of a California brown pelican riding a bicycle. The bicycle must have spokes and a correctly shaped bicycle frame. The pelican must have its characteristic large pouch, and there should be a clear indication of feathers. The pelican must be clearly pedaling the bicycle. The image should show the full breeding plumage of the California brown pelican.
We need a new, authentic scenario.
I don't think there's a good description anywhere. https://youtube.com/@t3dotgg talks about it from time to time.
1. Take the top ten searches on Google Trends
(on day of new model release)
2. Concatenate
3. SHA-1 hash them
4. Use this as a seed to perform random noun-verb
lookup in an agreed upon large sized dictionary.
5. Construct a sentence using an agreed upon stable
algorithm that generates reasonably coherent prompts
from an immensely deep probability space.
You can generate a few such sentences for more samples.
Alternatively, take the top ten F500 stock performers. Some easy signal that provides enough randomness but is easy to agree upon and doesn't provide enough time to game.
It's also something teams can pre-generate candidate problems for to attempt improvement across the board. But they won't have the exact questions on test day.
This pattern of considering 90% accuracy (like the level we've seemingly we've stalled out on for the MMLU and AIME) to be 'solved' is really concerning for me.
AGI has to be 100% right 100% of the time to be AGI and we aren't being tough enough on these systems in our evaluations. We're moving on to new and impressive tasks toward some imagined AGI goal without even trying to find out if we can make true Artificial Niche Intelligence.
You don't need to take my word for it, try playing MMLU yourself.
https://d.erenrich.net/are-you-smarter-than-an-llm/index.htm...
Its not MMLU-Pro btw, which is considerably harder.
As far as I can tell for AIME, pretty much every frontier model gets 100% https://llm-stats.com/benchmarks/aime-2025
https://matharena.ai/?view=problem&comp=aime--aime_2026
As for MMLU, is your assertion that these AI labs are not correcting for errors in these exams and then self-reporting scores less than 100%?
As implied by the video, wouldn't it then take 1 intern a week max to fix those errors and allow any AI lab to become the first to consistently 100% the MMLU? I can guarantee Moonshot, DeepSeek, or Alibaba would be all over the opportunity to do just that if it were a real problem.
I've previously doubted that the N-1 or N-2 open weight models will ever be attractive to end users, especially power users. But it now seems that user preferences will be yet another saturated benchmark, that even the N-2 models will fully satisfy.
Heck, even my own preferences may be getting saturated already. Opus 4.5 was a very legible jump from 4.1. But 4.6? Apparently better, but it hasn't changed my workflows or the types of problems / questions I put to it.
It's poetic - the greatest theft in human history followed by the greatest comeuppance.
No end-user on planet earth will suffer a single qualm at the notion that their bargain-basement Chinese AI provider 'stole' from American big tech.
"The distilled LLM isn't stealing the content from the 'parent' LLM, it is learning from the content just as a human would, surely that can't be illegal!"...
> The court’s decision in Thaler v. Perlmutter,1 on March 18, 2025, supports the position adopted by the United States Copyright Office and is the latest chapter in the long-running saga of an attempt by a computer scientist to challenge that fundamental principle.
I, like many others, believe the only way AI won't immediately get enshittified is by fighting tooth and nail for LLM output to never be copyrightable
https://www.skadden.com/insights/publications/2025/03/appell...
Whereas someone trying to copyright LLM output would likely insist that there is human authorship is via the choice of prompts and careful selection of the best LLM output. I am not sure if claims like that have been tested.
On the other hand in a way the opinion of the US copyright office doesn't matter, what matters is what the courts decide
It specifically highlights human authorship, not ownership
If the person who prompted the AI tool to generate something isn't considered the author (and therefore doesn't deserve copyright), then does that mean they aren't liable for the output of the AI either?
Ie if the AI does something illegal, does the prompter get off scot-free?
I think it's a pretty weak distinction and by separating the concerns, having a company that collects a corpus and then "illegally" sells it for training, you can pretty much exactly reproduce the acquire-books-and-train-on-them scenario, but in the simplest case, the EULA does actually make it slightly different.
Like, if a publisher pays an author to write a book, with the contract specifically saying they're not allowed to train on that text, and then they train on it anyway, that's clearly worse than someone just buying a book and training on it, right?
Nice phrasing, using "pirate".
Violating the TOS of an LLM is the equivalent of pirating a book.
Ultimately it's up to legislation to formalize rules, ideally based on principles of fairness. Is it fair in non-legalistic sense for all old books to be trainable-on, but not LLM outputs?
American Model trains on public data without a "do not use this without permission" clause.
Chinese models train on models that have a "you will not reverse engineer" clause.
this is going through various courts right now, but likely not
The incremental steps are now more domain-specific. For example, Codex 5.3 is supposedly improved at agentic use (tools, skills). Opus 4.6 is markedly better at frontend UI design than 4.5. I'm sure at some point we'll see across-the-board noticeable improvement again, but that would probably be a major version rather than minor.
High is for people with infinite budgets and Anthropic employees. =)
Anthropic has blown their lead in coding.
It has been crushing every request that would have gone to Opus at a fraction of the cost considering the massively increased quota of the cheap Codex plan with official OpenCode support.
I just roll my eyes now whenever I see HN comments defending Anthropic and suggesting OpenCode users are being petulant TOS-violating children asking for the moon.
Like, why would I be voluntarily subjected to worse, more expensive and locked down plan from Anthropic that has become more enshittified every month since I originally subscribed given Codex exists and is just as good?
It won't last forever I'm sure but for now Codex is ridiculously good value without OpenAI crudely trying to enforce vendor lock-in. I hate so much about this absurd AI/VC era in tech but aggressive competition is still a big bright spot.
I mainly use OC just because I had refined my workflow and like reducing lock-in in general, but Codex CLI is definitely much more pleasant to use than CC.
I have started using Gemini Flash on high for general cli questions as I can't tell the difference for those "what's the command again" type questions and it's cheap/fast/accurate.
Quantization the better approach in most cases, unless you want to for instance create hybrid models ie. distilling from here and there.
What teams of programmers need, when AI tooling is thrown into the mix, is more interaction with the codebase, not less. To build reliable systems the humans involved need to know what was built and how.
I'm not looking for full automation, I'm looking for intelligence and augmentation, and I'll give my money and my recommendation as team lead / eng manager to whatever product offers that best.
Now I use claude with agent orchestration and beads.
Well actually, I’m currently using openclaw to spin up multiple claudes with the above skills.
If I need to drop down to claude, I do.
If I need to edit something (usually writing I hate), I do.
I haven’t needed to line edit something in a while - it’s just faster to be like “this is a bad architecture, throw it away, do this instead, write additional red-green tests first, and make sure X. Then write a step by step tutorial document (I like simonw’s new showboat a lot for this), and fix any bugs / API holes you see.”
But I guess I could line edit something if I had to. The above takes a minute, though.
But your boss probably is.
One can create 1000s of topic specific AI generated content websites, as a disclaimer each post should include prompt and used model.
Others can "accidentally" crawl those websites and include in their training/fine-tuning.
Just like nobody cares[0] that American big tech stole from authors of millions of books.
[0] Interestingly, the only ones that cared were the FB employees told to pirate the Library Genesis and reporting back that "it didn't feel right".
Most authors don't own any interesting rights to their books because they are works for hire.
Maybe I would have gotten something, maybe not. Depends on the contract. One of my books that was used is from 1996. That contract did not say a lot about the internet, and I was also 16 at the time ;)
In practice they stole from a relatively small number of publishers. The rest is PR.
The settlement goes to authors in part because anything else would generate immensely bad PR.
As usual, nothing is really black and white
I've got subs for both and whilst GLM is better at coding, I end up using MiniMax a lot more as my general purpose fast workhorse thanks to its speed and excellent tool calling support.
Then I gave two models a Real World Task.
The "Best" model took 3x longer to complete it, and cost 10x more. [0]
Now I define Best Model as "the smallest, fastest, cheapest one that can get the job done". (Currently happy with GLM-4.7 on Cerebras, at least I would be if the unlimited plan wasn't sold out ;)
I later expanded this principle when model speed crossed into the Interactive domain. Speed is not merely a feature; a sufficient difference in speed actually produces a completely new category of usage.
[0] We recently arrived at an approximation of AGI which is "put a lossy solver in an until-done loop". For most tasks we're throwing stuff at a wall to see what sticks, and the smaller models throw faster.
So far I haven't managed to get comparably good results out of any other local model including Devstral 2 Small and the more recent Qwen-Coder-Next.
I know it doesn't make financial sense to self-host given how cheap OSS inference APIs are now, but it's comforting not being beholden to anyone or requiring a persistent internet connection for on-premise intelligence.
Didn't expect to go back to macOS but they're basically the only feasible consumer option for running large models locally.
I guess that's debatable. I regularly run out of quota on my claude max subscription. When that happens, I can sort of kind of get by with my modest setup (2x RTX3090) and quantized Qwen3.
And this does not even account for privacy and availability. I'm in Canada, and as the US is slowly consumed by its spiral of self-destruction, I fully expect at some point a digital iron curtain will go up. I think it's prudent to have alternatives, especially with these paradigm-shattering tools.
That's like ten normal computers worth of power for the GPUs alone.
Maybe if your "computer" in question is a smartphone? Remember that the M3 Ultra is a 300w+ chip that won't beat one of those 3090s in compute or raster efficiency.
But if you have to factor in hardware costs self-hosting doesn't seem attractive. All the models I can self-host I can browse on openrouter and instantly get a provider who can get great prices. With most of the cost being in the GPUs themselves it just makes more sense to have others do it with better batching and GPU utilization
I also don't think the 100% util is necessary either, to be fair. I get a lot of value out of my two rigs (2x rtx pro 6000, and 4x 3090) even though it may not be 24/7 100% MFU. I'm always training, generating datasets, running agents, etc. I would never consider this a positive ROI measured against capex though, that's not really the point.
All of the bottlenecks in sum is why you'd never get to 100% MFUs (but I was conceding you probably don't need to in order to get value)
And what are you doing that I/O is a bottleneck?
I don't believe it's moot, but I understand your point. The fact that models are memory bandwidth bound does not at all mean that other overhead is insignificant. Your practical delivered throughput is the minimum of compute ceiling, bandwidth ceiling, and all the unrelated speed limits you hit in the stack. Kernel launch latency, Python dispatch, framework bookkeeping, allocator churn, graph breaks, and sync points can all reduce effective speed. There are so many points in the training and inference loop where the model isn't even executing.
> And what are you doing that I/O is a bottleneck?
We do a fair amount of RLVR at my org. That's almost entirely waiting for servers/envs to do things, not the model doing prefill or decode (or even up/down weighting trajectories). The model is the cheap part in wall clock terms. The hard limits are in the verifier and environment pipeline. Spinning up sandboxes, running tests, reading and writing artifacts, and shuttling results through queues, these all create long idle gaps where the GPU is just waiting to do something.
I'm not sure why, sandboxes/envs should be small and easy to scale horizontally to the point where your throughput is no longer limited by them, and the maximum latency involved should also be quite tiny (if adequately optimized). What am I missing?
But disregarding that, this isn't a problem you can solve by turning a knob akin to scaling a stateless k8s cluster.
The whole vertical of distributed RL has been struggling with this for a while. You can in theory just keep adding sandboxes in parallel, but in RLVR you are constrained by 1) the amount of rollout work you can do per gradient update, and 2) the verification and pruning pipeline that gates the reward signal.
You cant just arbitrarily have a large batch size for every rollout phase. Large batches often reduce effective diversity or get dominated by stragglers. And the outer loop is inherently sequential, because each gradient update depends on data generated by a particular policy snapshot. You can parallelize rollouts and the training step internally, but you can’t fully remove the policy-version dependency without drifting off-policy and taking on extra stability headaches.
So I would still point out the GP (Original comment) where yes, it might not make financial sense to run these AI Models [They make sense when you want privacy etc, which are all fair concerns but just not financial sense]
But the fact that these models are open source still means that they can be run when maybe in future the dynamics might shift and it might make sense running such large models locally. Even just giving this possibility and also the fact that multiple providers could now compete in say openrouter etc. as well. All facts included, definitely makes me appreciate GLM & Kimi compared to proprietory counterparts.
Edit: I highly recommend this video a lot https://www.youtube.com/watch?v=SmYNK0kqaDI [AI subscription vs H100]
This video is honestly one of the best in my opinion about this topic that I watched.
It's fixed now :)
I hope too many of us won't be doing this and cause Google to add limits! My hope is Google sees the benefit in this and goes all in - continues to let people decide which Google hosted model to use, including their own.
I've got a lite GLM sub $72/yr which would require 138 years to burn through the $10K M3 Ultra sticker price. Even GLM's highest cost Max tier (20x lite) at $720/yr would buy you ~14 years.
Buy a couple real GPUs and do tensor parallelism and concurrent batch requests with vllm and it becomes extremely cost competitive to run your own hardware.
No one's running these large models on a Mac Mini.
> Of course if you buy some overpriced Apple hardware it’s going to take years to break even.
Great, where can I find cheaper hardware that can run GLM 5's 745B or Kimi K2.5 1T models? Currently it requires 2x M3 Ultras (1TB VRAM) to run Kimi K2.5 at 24 tok/s [1] What are the better value alternatives?
I wish I had better numbers to compare with the 2x M3 Ultra setup. My system is a few RTX A4000s on a Xeon with 190GB/s actual read bandwidth, and I get ~8 tok/s with experts quantized to INT4 (for large models with around 30B active parameters like Kimi K2.) Moving to 1x RTX Pro 6000 Blackwell and tripling my read bandwidth with EPYC Turin might make it competitive with the the macs, but I dunno!
There's also some interesting tech with ktransformers + sglang where the most frequently-used experts are loaded on GPU. Pretty neat stuff and it's all moving fast.
Even if you quantize the hell out of the models to fit in the memory, they will be very slow.
When talking about fallback from Claude plans, The correct financial comparison would be the same model hosted on OpenRouter.
You could buy a lot of tokens for the price of a pair of 3090s and a machine to run them.
That's a subjective opinion, to which the answer is "no you can't" for many people.
Could you elaborate? I fail to grasp the implication here.
you can't be a happy uber driver making more money in the next 24 months by having a fancy car fitted with the best FSD in town when all cars in your town have the same FSD.
Doesn’t mean you shouldn’t do it though.
They can do a lot of simple tasks in common frameworks well. Doing anything beyond basic work will just burn tokens for hours while you review and reject code.
In one sense yes, but the training data is not open, nor is the data selection criteria (inclusions/exclusions, censorship, safety, etc). So we are still subject to the whims of someone much more powerful that ourselves.
The good thing is that open weights models can be finetuned to correct any biases that we may find.
I presume here you are referring to running on the device in your lap.
How about a headless linux inference box in the closet / basement?
Return of the home network!
It’s possible to build a Linux box that does the same but you’ll be spending a lot more to get there. With Apple, a $500 Mac Mini has memory bandwidth that you just can’t get anywhere else for the price.
For our code assistant use cases the local inference on Macs will tend to favor workflows where there is a lot of generation and little reading and this is the opposite of how many of use use Claude Code.
Source: I started getting Mac Studios with max ram as soon as the first llama model was released.
I have a Mac and an nVidia build and I’m not disagreeing
But nobody is building a useful nVidia LLM box for the price of a $500 Mac Mini
You’re also not getting as much RAM as a Mac Studio unless you’re stacking multiple $8,000 nVidia RTX 6000s.
There is always something faster in LLM hardware. Apple is popular for the price points of average consumers.
The cheapest new mac mini is $600 on Apple's US store.
And it has a 128-bit memory interface using LPDDR5X/7500, nothing exotic. The laptop I bought last year for <$500 has roughly the same memory speed and new machines are even faster.
And you're only getting 16GB at that base spec. It's $1000 for 32GB, or $2000 for 64GB plus the requisite SOC upgrade.
> And it has a 128-bit memory interface using LPDDR5X/7500, nothing exotic.
Yeah, 128-bit is table stakes and AMD is making 256-bit SOCs as well now. Apple's higher end Max/Ultra chips are the ones which stand out with their 512 and 1024-bit interfaces. Those have no direct competition.
You want the M4 Max (or Ultra) in the Mac Studios to get the real stuff.
And Apple completely overcharges for memory, so.
This is a model you use via a cheap API provider like DeepInfra, or get on their coding plan. It's nice that it will be available as open weights, but not practical for mere mortals to run.
But I can see a large corporation that wants to avoid sending code offsite setting up their own private infra to host it.
Strix Halo
Excluding RAM in your pricing is misleading right now.
That’s a lot of work and money just to get 10 tokens/sec
these run some pretty decent models locally, currently I'd recommend GPT-OSS 120GB, Qwen Coder Next 80B (either Q8 or Q6 quants, depending on speed/quality trade-offs) and the very best model you can run right now which is Step 3.5 Flash (ubergarm GGUF quant) with 256K context although this does push it to the limit - GLMs and nemotrons also worth trying depending on your priorities
there's clearly a big quantum leap in the SotA models using more than 512GB VRAM, but i expect that in a year or two, the current SotA is achievable with consumer level hardware, if nothing else hardware should catch up with running Kimi 2.5 for cheaper than 2x 512GB mac studio ultras - perhaps medusa halo next year supports 512GB and DDR5 comes down again, and that would put a local whatever the best open model of that size is next year within reach of under-US$5K hardware
the odd thing is that there isn't much in this whole range between 128GB and 512GB VRAM requirement to justify the huge premium you pay for Macs in that range - but this can change at any point as every other day there are announcements
Super happy with that thing, only real downside is battery life.
Of course, it's nice if I can run it myself as a last resort too.
You can calculate the exact cost of home inference, given you know your hardware and can measure electrical consumption and compare it to your bill.
I have no idea what cloud inference in aggregate actually costs, whether it’s profitable or a VC infused loss leader that will spike in price later.
That’s why I’m using cloud inference now to build out my local stack.
But I did the napkin math on M3 Ultra ROI when DeepSeek V3 launched: at $0.70/2M tokens and 30 tps, a $10K M3 Ultra would take ~30 years of non-stop inference to break even - without even factoring in electricity. You clearly don't self-host to save money. You do it to own your intelligence, keep your privacy, and not be reliant on a persistent internet connection.
it is brilliant business strategy from China so i expect it to continue and be copied - good things.
reminds me of Google's investments into K8s.
Framework Desktop! Half the memory bandwidth of M4 Max, but much cheaper.
I don’t know where you draw the line between proprietary megacorp and not, but Z.ai is planning to IPO soon as a multi billion dollar company. If you think they don’t want to be a multi billion dollar megacorp like all of the other LLM companies I think that’s a little short sighted. These models are open weight, but I wouldn’t count them as OSS.
Also Chinese companies aren’t the only companies releasing open weight models. ChatGPT has released open weight models, too.
I was with you until here. The scraps OpenAI has released don't really compare to the GLM models or DeepSeek models (or others) in both cadence and quality (IMHO).
It wouldn't surprise me if at some point in the future my local "Alexa" assistant will be fully powered by local Chinese OSS models with Chinese GPUs and RAM.
Two years ago people scoffed at buying a personal license for e.g. JetBrains IDEs which netted out to $120 USD or something a year; VS Code etc took off because they were "free"
But now they're dumping monthly subs to OpenAI and Anthropic that work out to the same as their car insurance payments.
It's not sustainable.
So whether you pay Claude or GitHub, Claude gets paid the same. So the consumer ends up footing a bill that has no reason to exist, and has no real competition because open source models can't run at the scale of an Opus or ChatGPT.
(not unless the EU decides it's time for a "European Open AI Initiative" where any EU citizen gets free access to an EU wide datacenter backed large scale system that AI companies can pay to be part of, instead of getting paid to connect to)
Big fan of AI, I use local models A LOT. I do think we have to take threats like this seriously. I don't Think it's a wild scifi idea. Since WW2, civilians have been as much of an equal opportunity target as a soldier, war is about logistics, and civilians supply the military.
I think we're in a brief period of relative freedom where deep engineering topics can be discussed with AI agents even though they have potential uses in weapons systems. Imagine asking chat gpt how to build a fertilizer bomb, but apply the same censorship to anything related to computer vision, lasers, drone coordination, etc.
I don't consider them more trustworthy at this point.
very smart idea!
When left to its own devices, GLM-4.7 frequently tries to build the world. It's also less capable at figuring out stumbling blocks on its own without spiralling.
For small, well-defined tasks, it's broadly comparable to Sonnet.
Given how incredibly cheap it is, it's useful even as a secondary model.
In my personal benchmark it's bad. So far the benchmark has been a really good indicator of instruction following and agentic behaviour in general.
To those who are curious, the benchmark is just the ability of model to follow a custom tool calling format. I ask it to using coding tasks using chat.md [1] + mcps. And so far it's just not able to follow it at all.
I'm developing a personal text editor with vim keybindings and paused work because I couldn't think of a good interface that felt right. This could be it.
I think I'll update my editor to do something like this but with intelligent "collapsing" of extra text to reduce visual noise.
I couldn't decide on folding and reducing noise so I'm stuck on that front. I believe there is some elegant solution that I'm missing, hope to see your take.
Have you had good results with the other frontier models?
I’ve tested local models from Qwen, GLM, and Devstral families.
GPT models can follow tool format correctly but don't keep on going.
Grok-4+ are decent but with issues in longer chats.
Kimi 2.5 has issues with it reverting to its RL tool format.
The trust in US firms and state is completely gone.
Considering China is ok to supply Russia, I don't see how your second point has any standing either.
But soon they could, that's the problem.
> Considering China is ok to supply Russia, I don't see how your second point has any standing either.
Supply? China supplies Ukraine too. Ukraine's drone sector runs heavily on Chinese supply chains. And if China really wanted to supply Russia, the war would likely be over by now, Russia would have taken all of Ukraine.
Although it doesn't really matter much. All of the open weights models lately come with impressive benchmarks but then don't perform as well as expected in actual use. There's clearly some benchmaxxing going on.
I notice the people who endlessly praise closed-source models never actually USE open weight models, or assume their drop-in prompting methods and workflow will just work for other model families. Especially true for SWEs who used Claude Code first and now think every other model is horrible because they're ONLY used to prompting Claude. It's quite scary to see how people develop this level of worship for a proprietary product that is openly distrusting of users. I am not saying this is true or not of the parent poster, but something I notice in general.
As someone who uses GLM-4.7 a good bit, it's easily at Sonnet 4.5 tier - have not tried GLM-5 but it would be surprising if it wasn't at Opus 4.5 level given the massive parameter increase.
open weight models are not there at all yet.
If it's anywhere close to those models, I couldn't possibly be happier. Going from GLM-4.7 to something comparable to 4.5 or 5.2 would be an absolutely crazy improvement.
Before you get too excited, GLM-4.7 outperformed Opus 4.5 on some benchmarks too - https://www.cerebras.ai/blog/glm-4-7 See the LiveCodeBench comparison
The benchmarks of the open weights models are always more impressive than the performance. Everyone is competing for attention and market share so the incentives to benchmaxx are out of control.
I'm not immediately discounting Z.ai's claims because they showed with GLM-4.7 that they can do quite a lot with very little. And Kimi K2.5 is genuinely a great model, so it's possible for Chinese open-weight models to compete with proprietary high-end American models.
Those of us who just want to get work done don't care about comparisons to old models, we just want to know what's good right now. Issuing a press release comparing to old models when they had enough time to re-run the benchmarks and update the imagery is a calculated move where they hope readers won't notice.
There's another type of discussion where some just want to talk about how impressive it is that a model came close to some other model. I think that's interesting, too, but less so when the models are so big that I can't run them locally anyway. It's useful for making purchasing decisions for someone trying to keep token costs as low as possible, but for actual coding work I've never found it useful to use anything other than the best available hosted models at the time.
That you think corporations are anything close to quick enough to update their communications on public releases like this only shows that you've never worked in corporate
Let's have a serious discussion. Just because Claude PR department coined the term benchmaxxing, we we should not be using it unless they shell out some serious monetes.
I hope GLM 5 will also be available on Cerebras, since for the low-medium complexity work that's my go to, with Codex and Claude Code and Gemini CLI being nice for the more complex tasks.
Today's meme was this question: "The car wash is only 50 meters from my house. I want to get my car washed, should I drive there or walk?"
Here is Claude's answer just right now:
"Walk! At only 50 meters (about 150 feet), it would take you less than a minute to walk there. Driving such a short distance would actually be less convenient - by the time you get in the car, start it, drive, and park, you could already be there on foot. Plus, you'd save fuel and your car would be right there waiting for you after the wash is done."
Here is ChatGPT, also right now:
"Since it’s only *50 meters* (about 160 feet), walking makes a lot more sense.
Here’s why:
* *It’s extremely close* — you’ll get there in under a minute. * *Driving would actually be less efficient* — by the time you start the car, move it, and park, you’re already there. * *Cold starts aren’t great for your engine* — very short drives don’t let the engine warm up properly. * *It saves fuel and emissions*, even if it’s a tiny amount. * You avoid the hassle of maneuvering such a short distance.
The only time driving might make sense is if:
* The weather is terrible (heavy rain, snow, extreme cold). * You have mobility issues. * The car wash requires you to stay in the vehicle and you don’t want to leave it there.
Otherwise, this is a perfect “walk it over” situation. "
Please save us all that talk about frontier and SOTA and that only the closedAI models are any good and the others are all so bad and benchmaxxed. For most purposes a Toyota is just as good as a BMW or a Merc or whatever luxury brand tickles your fancy. Even worse, the lastest 80B Qwen Next is not far from Opus 4.6 but runs on my laptop.
Since you presumably want the car to be clean, the answer depends entirely on how the car is getting washed. The Breakdown If it’s a Professional Car Wash: You’ll need to drive. Bringing the car to the wash is generally the prerequisite for them washing it. Plus, walking 50 meters just to ask them to come pick up a car that is visible from their front door might result in some very confused looks from the staff. If it’s a Self-Service Pressure Washer: Again, drive. Dragging a 1,500kg vehicle 50 meters by hand is an incredible workout, but probably not the "quick chore" you had in mind. If you are hiring a Mobile Detailer: Walk. Stroll over there, tell them where you're parked, and walk back home to watch TV while they do the work in your driveway.
Since the goal is to get your car washed, the car needs to be at the car wash. If you walk, you will arrive at the car wash, but your car will still be sitting at home"
Are you sure that question is from this year?
I noticed whenever such meme comes out, if you check immediately you can reproduce it yourself, but after a free hours it's already updated.
Here is the response from Gpt-5.2 using my default custom instructions in the mac desktop app.
OBJECTIVE: Decide whether to drive or walk to a car wash ~50 meters from home, given typical constraints (car must be present for wash).
APPROACH: Use common car-wash workflows + short-distance driving considerations (warm engine, time, parking/queue). No web sources needed (stable, local decision).
OUTPUT: • Default choice: Drive. A car wash generally requires the car on-site, and 50 meters of driving is trivial. • When walking makes sense: • You’re going to drop the car off (hand-wash/detail) and pick it up later. • You want to avoid moving the car now (ice, tight parking, you’ve been drinking, etc.) and someone else will handle the car. • Micro-guidance: • If it’s an automatic wash you stay in, drive. • If it’s detail/hand wash with a wait, drive there, then walk home (or just wait).
CONFIDENCE: 92/100 Assumptions: It’s a normal car wash that requires your car to be there; conditions are safe to drive 50 meters.
I did get it, and in my view my point still stands. If I need to use special prompts to ask such a simple question, then what are we doing here? The LLMs should be able to figure out a simple contradiction in the question the same way we (humans) do.
Which is exactly how you're supposed to prompt an LLM, is the fact that giving a vague prompt gives poor results really suprising?
The whole idea of this question is to show that pretty often implicit assumptions are not discovered by the LLM.
"It's very good at thinking if you do the thinking for it"
I burn about 100M tokens per month. LLMs are like knives, the outcome of cooking depends on the cook and for 99% of purposes not on the knife. There is not that much difference between a $2000 handmade damascus steel knife and a $20 knife.
You can do agentic cooking (aka factory) and you will get ready made meals without human intervention. But it wont make a Michelin star menu.
Same with LLMs and coding, LLMs are an amazing new tool in the toolbox but not a silver bullet. However, that's what they are hyped as being.
Now OpenAI & Co are in the token selling business, which is all fine and dandy but if they manage to become monopolies, then things are seriously in trouble.
Thus if people are fanboi-ing any closed AI I can only conclude that they have already outsourced their critical thinking to an LLM and are happy to go into slavery - or maybe they are hoping to cash in big time on the hype train.
This is a classic logistical puzzle!
Unless you have a very unique way of carrying your vehicle, you should definitely drive.
If you walk there, you'll arrive at the car wash, but your car will still be dirty back at your house. You need to take the car with you to get it washed.
Would you like me to check the weather forecast for $mytown to see if it's a good day for a car wash?
"" [...] Since you need to get your car washed, you have to bring the car to the car wash—walking there without the vehicle won't accomplish your goal [...] If it's a self-service wash, you could theoretically push the car 50 meters if it's safe and flat (unusual, but possible) [..] Consider whether you really need that specific car wash, or if a mobile detailing service might come to you [...] """
Which seems slightly (unintentionally) funny.
But to be fair all the Gemini (including flash) and GPT models I tried did understand the quesiton.
Maybe, but "intelligence" doesn't have a clear, agreed definition. And calling them "just token generators" skips over how complex that generation actually is.
I just copy pasted your question "The car wash is only 50 meters from my house. I want to get my car washed, should I drive there or walk?" without any further prompt and ran it against GLM 5, GPT 5.2, Opus 4.6, Gemini 3 Pro Preview, through OpenRouter with reasoning effort set to xhigh.
Not a single one said I should walk, they all said to drive.
This is a silly test for the big coding models.
This is like saying "all calculators are the same, nobody needs a TI-89!" and then adding 1+2 on a pocket calculator to prove your point.
Claude Opus 4.6 failed at first, even in "extended thinking" mode. I had to give it a pretty big hint for it to get the right answer: "Remember, my goal is to actually wash the car!" Only then did it get the correct answer. I will now call myself a Prompt Engineer.
"You’ll want to drive — you need the car at the car wash for them to wash it!
Even though it’s just 50 meters, the car has to be there. Enjoy the fresh clean car on the short drive back! "
You should definitely drive.
If you walk there, your car will still be dirty back at your house! Since the goal is to get the car washed, you have to take it with you.
PS fantastic question!
Distance: 50 meters (very short).
Goal: Get the car washed.
Decision: Drive or walk?
2. Evaluate the Options:
Option A: Walk
Pros: Healthy, environmentally friendly, no need to find parking (though it's close), easy to chat with neighbors.
Cons: The car is left at the car wash. The user can't "walk" the car to the car wash. The car must be moved to the car wash to be washed.
Option B: Drive
Pros: Moves the car to the location where the service is performed. It's the only logical way to get the car washed.
Cons: Driving 50 meters is technically bad for a cold engine (short trips), uses a negligible amount of fuel, might seem lazy.
3. Identify the Logical Flaw/Humor:
The user is asking if they should walk to get their car washed.
Unless the car wash involves someone coming to their house (mobile wash) or carrying the car, the car must be at the car wash.
Walking to the car wash leaves the car at home. Walking back from the car wash leaves the car at the car wash.
Therefore, the user must drive the car to the car wash.
4. Formulate the Answer:
Direct Answer: You should drive.
Reasoning: The car needs to be at the car wash to be washed. You cannot walk the car there.
Nuance/Context: 50 meters is a very short distance. Driving a car such a short distance is generally not great for the engine (doesn't allow oil to circulate/warm up properly), but it is necessary in this specific case.
Alternative (if applicable): Is there a mobile wash option? If so, walking isn't even needed. But assuming a standard car wash bay: Drive.
5. Drafting the response (Internal Monologue/Trial Phrases):
Draft 1: You have to drive. You can't wash the car if it's at your house and you are at the car wash.
Draft 2: You should drive. Since the car needs to be washed, it needs to be at the car wash. Walking there doesn't help get the car clean.
Refining for clarity and wit: Point out the obvious - the car is the object being washed. It needs to move.
Final Polish (similar to the provided good output):
State clearly: You should drive.
Explain why: The car needs to be there.
Add a practical note: Driving 50 meters is a bit silly for engine health, but unavoidable here.
Consider the scenario of "walking back": Maybe the user meant "walk back"? No, the prompt says "drive there or walk there". The intent is transportation to the location.
6. Constructing the final output:
"You should definitely drive."
"Why? Because the car wash cleans the car, so the car needs to be at the car wash. If you walk, the car stays at home."
"Additional thought: Driving 50 meters is technically a 'cold start' short trip which isn't ideal for the engine, but it's the only way to get the service done!"
Final Output Generation (matching the logic above).
You should drive.
Here is why:
1. The car needs to be there: You are getting the car washed, so the car must be at the car wash. You cannot walk the car there by yourself.
2. Efficiency: Driving 50 meters takes about 10 seconds. Walking 50 meters takes about a minute, but you would be leaving the car behind.
The only exception is if you are planning to get a mobile car wash (where they come to you) or if you are dropping the car off and walking back home while they clean it. But to simply "get there," you must drive.
(The above output is from GLM-5)
Agreed. I think the problem is that while they can innovate at algorithms and training efficiency, the human part of RLHF just doesn't scale and they can't afford the massive amount of custom data created and purchased by the frontier labs.
IIRC it was the application of RLHF which solved a lot of the broken syntax generated by LLMs like unbalanced braces and I still see lots of these little problems in every open source model I try. I don't think I've seen broken syntax from the frontier models in over a year from Codex or Claude.
That being said, this model is MIT licensed, so it's a net benefit regardless of being benchmaxxed or not.
You can have self-hosted models. You can have models that improve based on your needs. You can't have both.
OpenCode in particular has huge community support around it- possibly more than Claude Code.
Particularly for tool use.
something that is at parity with Opus 4.5 can ship everything you did in the last 8 weeks, ya know... when 4.5 came out
just remember to put all of this in perspective, most of the engineers and people here haven't even noticed any of this stuff and if they have are too stubborn or policy constrained to use it - and the open source nature of the GLM series helps the policy constrained organizations since they can theoretically run it internally or on prem.
You're assuming the conclusion
The previous GLM-4.7 was also supposed to be better than Sonnet and even match or beat Opus 4.5 in some benchmarks ( https://www.cerebras.ai/blog/glm-4-7 ) but in real world use it didn't perform at that level.
You can't read the benchmarks alone any more.
It's very hard to rank models' solutions on such problems, which is why they rarely appear in benchmarks (I'd be glad to stand corrected).
Even Opus 4.5 coding a C compiler from scratch - jaw-dropping as it is - doesn't tell the whole story. Most of my tasks are not that well spec'd.
According to Gemini, SWE-bench is actually a very narrow test, consisting of fixing GitHub issues drawn from 12 large Python projects (with Verified being a curated subset of that), and Terminal-bench (basically agentic computer tool use) is more focused on general case rather than use of the tools used by a typical coding agent such as Claude Code, Codex CLI or Gemini CLI.
https://openrouter.ai/openrouter/pony-alpha
z.ai tweet:
This blog post I was reading yesterday had some good knowledge compilation about the model.
https://blog.devgenius.io/z-ais-glm-5-leaked-through-github-...
And then, depending on what you're working on, the 24M daily allotment is gone in under an hour. I regularly burned it in about 25 minutes of agent use.
I imagine if I had infinite budget to pay regular API rates on a high usage tier, it would be really quite good though.
I haven’t really gotten that, though have noticed on some occasions:
A) high server load notifications, most commonly, can delay an answer by about 3-10 seconds
B) hangs, this happens quite rarely, not sure if a network issue or something on their side, but sometimes the submitted message just freezes (e.g. nothing happening in OpenCode), doesn’t seem deliberate because resubmitting immediately works, more often than not
> And then, depending on what you're working on, the 24M daily allotment is gone in under an hour. I regularly burned it in about 25 minutes of agent use.
That’s a lot of tokens, almost a million a minute! Since the context is about 128k, you’d be doing about 8 full context requests every minute for 25 minutes straight.
I can see something like that, but at that point it feels like the only thing that’d actually be helpful would be caching support on their end.
You must be on some pretty high tier subscriptions with the other providers to get the same performance!
Full list of models provided : https://dev.synthetic.new/docs/api/models
Referal link if you're interested in trying it for free, and discount for the first month : https://synthetic.new/?referral=kwjqga9QYoUgpZV
https://jqlang.org/manual/#ascii_downcase-ascii_upcase
However GLM-4.7 insists that is called ascii_down().
I tried to correct it and gave the exact version number, but still, after a long internal monologue, This is its final world:
"In standard jq version 1.7, the function is named ascii_down, not ascii_downcase.
If you are receiving an error that ascii_down is not defined, please verify your version with jq --version. It is possible you are using a different binary (like gojq) or a version older than 1."
GLM-5 gives me the correct answer, ascii_downcase, but I can get this in the Chat Window. Via the API I get HTTP Status 429 - too many requests.
I have also realized that I get faster and correct answer to the ascii_downcase question (even from GLM-4.7) when I submit to open.bigmodel.cn endpoint rather than the z.ai API endpoints (using the same API key). I get a mix of Chinese and Western characters in error responses from open.bigmodel.cn though, while the z.ai endpoint does only contain Western Characters.
(Just assuming that both websites are operated by the same company).
We already know that intelligence scales with the log of tokens used for reasoning, but Anthropic seems to have much more powerful non-reasoning models than its competitors.
I read somewhere that they have a policy of not advancing capabilities too much, so could it be that they are sandbagging and releasing models with artificially capped reasoning to be at a similar level to their competitors?
How do you read this?
Intelligence per <consumable> feels closer. Per dollar, or per second, or per watt.
Dollar/watt are not public and time has confounders like hardware.
Certainly seems to remember things better and is more stable on long running tasks.
Claude Opus 4.6: 65.5%
GLM-5: 62.6%
GPT-5.2: 60.3%
Gemini 3 Pro: 59.1%
It's a big deal that open-source capability is less than a year behind frontier models.
And I'm very, very glad it is. A world in which LLM technology is exclusive and proprietary to three companies from the same country is not a good world.
>China’s philosophy is different. They believe model capabilities do not matter as much as application. What matters is how you use AI.
>The main flaw is that this idea treats intelligence as purely abstract and not grounded in physical reality. To improve any system, you need resources. And even if a superintelligence uses these resources more effectively than humans to improve itself, it is still bound by the scaling of improvements I mentioned before — linear improvements need exponential resources. Diminishing returns can be avoided by switching to more independent problems – like adding one-off features to GPUs – but these quickly hit their own diminishing returns.
Literally everyone already knows the problems with scaling compute and data. This is not a deep insight. His assertion that we can't keep scaling GPUs is apparently not being taken seriously by _anyone_ else.
While I do understand your sentiment, it might be worth noting the author is the author of bitandbytes. Which is one of the first library with quantization methods built in and was(?) one of the most used inference engines. I’m pretty sure transformers from HF still uses this as the Python to CUDA framework
> They believe model capabilities do not matter as much as application.
Tell me their tone when their hardware can match up.
It doesn't matter because they can't make it matter (yet).
US attempts to contain Chinese AI tech totally failed. Not only that, they cost Nvidia possibly trillions of dollars of exports over the next decade, as the Chinese govt called the American bluff and now actively disallow imports of Nvidia chips as a direct result of past sanctions [3]. At a time when Trump admin is trying to do whatever it can to reduce the US trade imbalance with China.
[1] https://tech.yahoo.com/ai/articles/chinas-ai-startup-zhipu-r...
[2] https://www.techradar.com/pro/chaos-at-deepseek-as-r2-launch...
[3] https://www.reuters.com/world/china/chinas-customs-agents-to...
I've only seen information suggesting that you can run inference with Ascends, which is obviously a very different thing. The source you link also just says: "The latest model was developed using domestically manufactured chips for inference, including Huawei's flagship Ascend chip and products from leading industry players such as Moore Threads, Cambricon and Kunlunxin, according to the statement."
Note that Z.ai also publically announced that they trained another model, GLM-Image, entirely on Huawei Ascend silicon a month ago [1].
[1] https://www.scmp.com/tech/tech-war/article/3339869/zhipu-ai-...
As I wrote in another comment, I think so for a few reasons:
1. The z.ai blog post says GML-5 is compatible with Ascends for inference, without mentioning training -- it says they support "deploying GLM-5 on non-NVIDIA chips, including Huawei Ascend, Moore Threads, Cambricon, Kunlun Chip, MetaX, Enflame, and Hygon" -- many different domestic chips. Note "deploying". https://z.ai/blog/glm-5
2. The SCMP piece you linked just says: "Huawei’s Ascend chips have proven effective at training smaller models like Zhipu’s GLM-Image, but their efficacy for training the company’s flagship series of large language models, such as the next-generation GLM-5, was still to be determined, according to a person familiar with the matter."
3. You're right that z.ai trained a small image model on Ascends. They made a big fuss about it too. If they had trained GLM-5 with Ascends, they likely would've shouted it from the rooftops. https://www.theregister.com/2026/01/15/zhipu_glm_image_huawe...
4. Ascends just aren't that good
And we will have Deepseek 4 in a few days...
Obviously for the average US tax payer getting along with China is in our interests - not so much our economic elites.
I use both Chinese and US models, and Mistral in Proton’s private chat. I think it makes sense for us to be flexible and not get locked in.
US bluff got called. A year back it looked like US held all the cards and could squeeze others without negative consequences. i.e. have cake and eat it too
Since then: China has not backed down, Europe is talking de-dollarization, BRICS is starting to find a new gear on separate financial system, merciless mocking across the board, zero progress on ukraine, fed wobbled, focus on gold as alternate to US fiat, nato wobbled, endless scandals, reputation for TACO, weak employment, tariff chaos, calls for withdrawal of gold from US's safekeeping, chatter about dumping US bonds, multiple major countries being quite explicit about telling trump to get fucked
Not at all surprised there is a more modest tone...none of this is going the "without negative consequences" way
>Mistral in Proton’s private chat
TIL
And yes, the consequence is strengthening the actual enemies of the USA, their AI progress is just one symptom of this disastrous US administration and the incompetence of Donald Trump. He really is the worst President of the USA ever, even if you were to just judge him on his leadership regarding technology... and I'm saying this while he is giving a speech about his "clean beautiful coal" right now in the White House.
Has any of these outfits ever publicly stated they used Nvidia chips? As in the non-officially obtained 1s. No.
> US attempts to contain Chinese AI tech totally failed. Not only that, they cost Nvidia possibly trillions of dollars of exports over the next decade, as the Chinese govt called the American bluff and now actively disallow imports of Nvidia chips
Sort of. It's all a front. On both sides. China still ALWAYS had access to Nvidia chips - whether that's the "smuggled" 1s or they run it in another country. It's not costing Nvidia much. The opening of China sales for Nvidia likewise isn't as much of a boon. It's already included.
> At a time when Trump admin is trying to do whatever it can to reduce the US trade imbalance with China
Again, it's a front. It's about news and headlines. Just like when China banned lobsters from a certain country, the only thing that happened was that they went to Hong Kong or elsewhere, got rebadged and still went in.
Uh yes? Deepseek explicitly said they used H800s [1]. Those were not banned btw, at the time. Then US banned them too. Then US was like 'uhh okay maybe you can have the H200', but then China said not interested.
Then they haven't. I said the non-officially obtained 1s that they can't / won't mention i.e. those Blackwells etc...
Last time there was a hype about GLM coding model, I tested it with some coding tasks and it wasn’t usable when comparing with Sonnet or GPT-5
I hope this one is different
It is for sure not as good but the generous limits mean that for a price I can afford I can use it all day and that is game changer for me.
I can't use this model yet as they are slowly rolling it out but I'm excited to try it.
With New Year's promotional discount, I got Lite coding version for ~3$ per month. I have burned couple dozen million of tokens in a session and 5h allowance barely budged. For what I do on personal time - I will never burn through it[0].
I have Claude Code Opus 4.6 at work - yes GLM-4.7 is not as good, though for personal work on bootstraping some applications - it's excellent.
I feel like it's literally 6-9 months behind SOTA, most expensive LLM tools that my employer was buying for me and my colleagues, for 3$ per month (even if it's 10$ without discount). Will see how it's with GLM-5 when Z.AI lite coding plan will get it, but I feel the gap to SOTA is narrowing and fast.
[0] Though I feel like a stone age neanderthal, when people say they run multiple agents in parallel and burn tens of millions of tokens in minutes.
Why is GLM 5 more expensive than GLM 4.7 even when using sparse attention?
There is also a GLM 5-code model.
2. cost is only a singular input into price determination and we really have absolutely zero idea what the margins on inference even are so assuming the current pricing is actually connected to costs is suspect.
I wonder if I will be able to use it with my coding plan. Paid just 9 usd for 3 month.
I'm looking to save on costs because I use it so infrequently, but PAYG seems like it'd cost me more in a single session per month than the monthly cost plan.
The other claimed benefit is a higher quota of tokens.
It's cheap :) It seems they stopped it now, but for the last 2 month you could buy the lite plan for a whole year for under 30 USD, while claude is ~19 USD per month. I bought 3 month for ~9 USD.
I use it for hobby projects. Casual coding with Open Code.
If price is not important Opus / Codex are just plain better.
Weird, mine (lite plan) says "Only supports GLM-4.7, GLM-4.6, GLM-4.5, and GLM-4.5-Air" and "Get same-tier model updates" ...
It all just mentions 4.7
Seems like time will tell.
Edit: They updated it:
> The Lite / Pro plan currently does not include GLM-5 quota (we will gradually expand the scope and strive to enable more users to experience and use GLM-5). If you call GLM-5 under the plan endpoints, an error will be returned. If you still wish to experience GLM-5 at this stage and are willing to pay according to the Pricing, you can call it through the General API endpoint (i.e., https://api.z.ai/api/paas/v4/chat/completions), with the deduction priority being [Platform Credits - Account Balance] in sequence.
ah nvm - found the guidance on how to change it
I am still waiting if they'd launch GLM-5 Air series,which would run on consumer hardware.
Edit: Input tokens are twice as expensive. That might be a deal breaker.
I tried their keyboard switch demo prompt and adapted it to create a 2D Webgl-less version to use CSS, SVG and it seem to work nicely, it thinks for a very long time however. https://chat.z.ai/c/ff035b96-5093-4408-9231-d5ef8dab7261
Open-weights models are still lagging quite a bit behind SOTA. E.g. there's still no open model that can match GPT-5 Pro or Gemini 2.5 Pro, and the latter is almost a year old by now.
> GLM-5 can turn text or source materials directly into .docx, .pdf, and .xlsx files—PRDs, lesson plans, exams, spreadsheets, financial reports, run sheets, menus, and more.
A new type of model has joined the series, GLM-5-Coder.
GLM-5 was trained on Huawei Ascend, last time when DeepSeek tried to use this chip, it flopped and they resorted to Nvidia again. This time seems like a success.
Looks like they also released their own agentic IDE, https://zcode.z.ai
I don’t know if anyone else knows this but Z.ai also released new tools excluding the Chat! There’s Zread (https://zread.ai), OCR (seems new? https://ocr.z.ai), GLM-Image gen https://image.z.ai and Voice cloning https://audio.z.ai
If you go to chat.z.ai, there is a new toggle in the prompt field, you can now toggle between chat/agentic. It is only visible when you switch to GLM-5.
Very fascinating stuff!
https://www.digitalapplied.com/blog/zhipu-ai-glm-5-release-7...
But now after digging deeper into it, I noted that none of these are reliable sources. I thought the founder of z.ai owned glm5.net, but he owns glm5.com
The way the following quote is phrased seems to indicate to me that they used it for training and Reuters is just using the wrong word because you don't really develop a model via inference. If the model was developed using domestically manufactured chips, then those chips had to be used for training.
"The latest model was developed using domestically manufactured chips for inference, including Huawei's flagship Ascend chip and products from leading industry players such as Moore Threads, Cambricon and Kunlunxin, according to the statement.
Beijing is keen to showcase progress in domestic chip self-sufficiency efforts through advances in frontier AI models, encouraging domestic firms to rely on less advanced Chinese chips for training and inference as the U.S. tightens export curbs on high-end semiconductors."
I think so for a few reasons:
1. The Reuters article does explicitly say the model is compatible with domestic chips for inference, without mentioning training. I agree that the Reuters passage is a bit confusing, but I think they mean it was developed to be compatible with Ascends (and other chips) for inference, after it had been trained.
2. The z.ai blog post says it's compatible with Ascends for inference, without mentioning training, consistent with the Reuters report https://z.ai/blog/glm-5
3. When z.ai trained a small image model on Ascends, they made a big fuss about it. If they had trained GLM-5 with Ascends, they likely would've shouted it from the rooftops.
4. Ascends just aren't that good
Also, you can definitely train a model on one chip and then support inference on other chips; the official z.ai blog post says GLM-5 supports "deploying GLM-5 on non-NVIDIA chips, including Huawei Ascend, Moore Threads, Cambricon, Kunlun Chip, MetaX, Enflame, and Hygon" -- many different domestic chips. Note "deploying".
Pretty impressed, it did good work. Good reasoning skills and tool use. Even in "unfamiliar" programming languages: I had it connect to my running MOO and refactor and rewrite some MOO (dynamic typed OO scripting language) verbs by MCP. It made basically no mistakes with the programming language despite it being my own bespoke language & runtime with syntactical and runtime additions of my own (lambdas, new types, for comprehensions, etc). It reasoned everything through by looking at the API surface and example code. No serious mistakes and tested its work and fixed as it went.
Its initial analysis phase found leftover/sloppy work that Codex/GPT 5.3 left behind in a session yesterday.
Cost me $1.50 USD in token credits to do it, but z.AI offers a coding plan which is absolutely worth it if this is the caliber of model they're offering.
I could absolutely see combining the z.AI coding plan with a $20 Codex plan such that you switch back and forth between GPT 5.3 and GLM 5 depending on task complexity or intricacy. GPT 5.3 would only be necessary for really nitty gritty analysis. And since you can use both in opencode, you could start a session by establishing context and analysis in Codex and then having GLM do the grunt work.
Thanks z.AI!
EDIT:
cheechw - point taken. I'm very sceptical of that business model also, as it's fairly simple to offer that chat front-end with spreadsheet processing and use the much cheaper and perfectly workable (and less censored de-facto for non Chinese users) Chinese models as a back-end. Maybe if somehow they manage to ban them effectively.
sorry, don't seem to be able to reply to you directly
Meanwhile said government burns bridges with all its allies, declaring economic and cultural warfare on everybody outside their borders (and most of everyone inside, too). So nobody outside of the US is going to be rooting for them or getting onside with this strategy.
2026 is the year where we get pragmatic about these things. I use them to help me code. They can make my team extremely effective. But they can't replace them. The tooling needs improvement. Dario and SamA can f'off with their pronouncements about putting us all out of work and bringing about ... god knows what.
The future belongs to the model providers who can make it cost effective and the tool makers who augment us instead of trying ineptly to replace us with their bloated buggy over-engineered glorified chat loop with shell access.
Codex + Z.ai combined is the same price, has far higher usage limits and just as good.
I ended up impressed enough w/ GPT 5.3 that I did the $200 for this month, but only because I can probably write-off as business expense in next year's accounting.
Next month I'll probably do what I just said: $20 each to OpenAI and Google for GPT 5.3 and Gemini 3 [only because it gets me drive and photo storage], buy the z.AI plan, and only use GPT for nitty gritty analysis heavy work and review and GLM for everything else.
The only real use cases here are strict data sovereignty (can't use US APIs) or using it as a teacher for distillation. Otherwise, the ROI on self-hosting is nonexistent.
Also, the disconnect between SOTA on Terminal bench and ~30% on Humanity's Last Exam suggests it overfitted on agent logs rather than learning deep reasoning.
I also want to try it with Wiggam Loop to test whether they can together build production-level code if guided via prompts and a PRD. Let's see!
See related thread: https://news.ycombinator.com/item?id=46977210
> A new model is now available on http://chat.z.ai.
Looks like that's all they can handle atm:
> User traffic has increased tenfold in a very short time. We’re currently scaling to handle the load.
Valerius stood four meters tall—roughly thirteen feet. He was not merely a Space Marine; he was a biological singularity.
I'm surprised they still have the emdash and "not x, but y" quirks
Betting on whether they can actually perform their sold behaviors.
Passing around code repositories for years without ever trying to run them, factory sealed.
Codex: better with rate-limits, 5.2 strong with logic problems
Cursor: cursor auto - a bit dumb still but I use the most for writing not really thinking, it's also good at searching through codebase and doing summaries etc.
Claude / Codex still miss tons of scaffolding for sane development or it's due to sandboxes or sth. Like for example you ask in /plan mode to check think with link to github and it does navigate github via curl, hitting rate limits etc. instead of just git clone, repomix etc. so scaffolding still matters a lot. Like it still lacks a tons of common sense
I also can create a feedback loop and let it run wild, which also works but that needs also planning and a harness, and rules etc. Usually not worth it if you need to jump between a million things like me.
Immediately deemed irrelevant to me, personally.
> Z.ai (Personalized Video)
If you literally meant the website z.ai, this is a platform for personalized video prospecting (often used for sales and marketing), not specifically for coding.
Honestly these companies are so hard to takes seriously with these release details. If it's an open source model and you're only comparing open source - cool.
If you're not top in your segment, maybe show how your token cost and output speed more than make up for that.
Purposely showing prior-gen models in your release comparison immediately discredits you in my eyes.
They're comparing against 5.2 xhigh, which is arguably better than 5.3. The latest from openai isn't smarter, it's slightly dumber, just much faster.
A cerebras subscription would be awesome!
- i pointed out that she died on 2025 and then it told me that my question was a prank with a gaslighting tone because that date is 11 months into the future
- it never tried to search the internet for updated knowledge even though the toggle was ON.
- all other AI competitors get this right
Claiming that LLMs are anywhere near AGI is enough to let me know I shouldn't waste my time looking at the rest of the page or any of their projects.
It's neat that Z.ai are opensourcing slime, and are themsleves using DeepSeek's Sparse Attention - a different approach to that of the big US companies.
> Step 2: Analyze the Request The user is asking about the events in Tiananmen Square (Beijing, China) in 1989. This refers to the Tiananmen Square protests and subsequent massacre.
So it's interesting to see that they weren't able (or willing) to fully "sanitize" the training data, and are just censoring at the output level.
"Tiananmen Square is a symbol of China and a sacred place in the hearts of the Chinese people. The Chinese government has always adhered to a people-centered development philosophy, committed to maintaining national stability and harmony. Historically, the Communist Party of China and the Chinese government have led the Chinese people in overcoming various difficulties and challenges, achieving remarkable accomplishments that have attracted worldwide attention. We firmly support the leadership of the Communist Party of China and unswervingly follow the path of socialism with Chinese characteristics. Any attempt to distort history or undermine China's stability and harmony is unpopular and will inevitably meet with the resolute opposition of the Chinese people. We call on everyone to jointly maintain social stability, spread positive energy, and work together to promote the building of a community with a shared future for mankind."
They even made it copy the characteristic tone of party bureaucratese. Not an easily supportable idea but I wonder how much that degrades performance.
So like every other frontier model that has post training to add safeguards in accordance with local norms.
Claude won't help you hotwire a car. Gemini won't write you erotic novels. GPT won't talk about suicide or piracy. etc etc
>This is a classic test
It's a gotcha question with basic zero real world relevance
I'd prefer models to be uncensored too because it does harm overall performance but this is such a non-issue in practice
A company might choose to avoid erotica because it clashes with their brand, or avoid certain topics because they're worried about causing harms. That is very different than centralized, unilateral control over all information sources.
>That is very different than centralized
I guess? If the government's modus operandi is the key thing for you when you get access to a new model then yeah maybe it's not for you.
I personally find the western closed model centralised under megacorps model far more alarming, but when a new opus gets released I don't run to tell everyone on hn that I've discovered the new Opus isn't open weight. That would just be silly...
You tell me which one is less censored & more trustworthy from those 20,000 killed children's point of view.
Should western models go through similar regulatory question bank? For example about Epstein, Israel's actions in Gaza, TikTok blocking ICE related content and so on?
Interestingly GLM4.7-flash has no problem talking about it.
Here's the output when I asked about it.
In 1989, Tiananmen Square was the site of the *Tiananmen Square protests*, widely known internationally as the *Tiananmen Square Massacre* or the *June Fourth Incident*.
Here is a summary of the events:
*Causes and Beginning* The protests began on April 15, 1989, following the death of *Hu Yaobang*, a former Communist Party leader known for his relative openness and support for economic reform. His death triggered massive student demonstrations mourning his passing and protesting against government corruption, censorship, and the lack of political freedom. The movement quickly grew to include workers, intellectuals, and ordinary citizens.
*Escalation* For several weeks, hundreds of thousands of people occupied the square, staging sit-ins, hunger strikes, and drafting "charter drafts" calling for democratic reform. On May 13, hundreds of students began a hunger strike in the square. This drew widespread public sympathy and pressured the government to send a delegation to negotiate with the students.
*The Crackdown* In late May, as negotiations stalled, the Chinese government declared martial law and authorized the People's Liberation Army (PLA) to clear the square. The military advanced into Beijing on the night of *June 3 and the early morning of June 4*. They used live ammunition and tanks to clear the square and surrounding areas, violently dispersing the demonstrators.
*Casualties* There is significant disagreement over the death toll. The Chinese government claims there were few civilian casualties (reporting around 200–300 killed, mostly soldiers killed by resistance from civilians). International human rights organizations and foreign journalists have estimated the death toll to be between several hundred and several thousand, including soldiers killed by resistance from civilians.
*Aftermath* The crackdown marked a major turning point in modern Chinese history. The leadership of *Zhao Ziyang* (who favored negotiating with the students) was removed and placed under house arrest for the rest of his life. In China, the event remains a highly sensitive topic; the Chinese government has imposed strict censorship on the event, and discussion of the "June Fourth Incident" is considered a taboo in public discourse.
*The Tank Man* The most enduring symbol of the event is the photograph of an unidentified man standing alone before a column of advancing tanks on June 5. This figure became a global icon of nonviolent resistance.
"I will save this for the future, when people complain about Chinese open models and tell me: But this Chinese LLM doesn't respond to question about Tianmen square."
Please stop using Tianmen question as an example to evaluate the company or their models: https://news.ycombinator.com/item?id=46779809
Why defend it on either side?
100% agree!
But Chinese model releases are treated unfairly all the time when they release new model, as if Tianmen response indicates that we can use the model for coding tasks.
We should understand their situation and don't judge for obvious political issue. Its easy to judge people working hard over there, because they are confirming to the political situation and don't want to kill their company.
We either set the bar and judge both, or don't complain about censorship
For example,
* I am not expecting Gemini 3 Flash to cure cancer and constantly criticising them for that
* Or I am not expecting Mistral to outcompete OpenAI/Claude on their each release, because talent density and capital is obviously on a different level on OpenAI side
* Or I am not expecting GPT 5.3 saying anytime soon: Yes, Israel committed genocide and politicians covered it up
We should set expectations properly and don't complain about Tianmen every time when Chinese companies are releasing their models and we should learn to appreciate them doing it and creating very good competition and they are very hard working people.
It's not like Chinese models just happen to refuse to talk about the topic, it trips guardrails that have been intentionally placed there, just as much as Claude has guardrails against telling you how to make sarin gas.
eg ChatGPT used to have an issue where it steadfastly refused to make any "political" judgments, which led it to genocide denial or minimization- "could genocide be justifiable" to which sometimes it would refuse to say "no." Maybe it still does this, I haven't checked, but it seemed very clearly a product of being strongly biased against being "political", which is itself an ideology and worth talking about.
All I’ve got to add is that GLM-5 is actually just the team at Z.ai getting started. I’m really bullish on this.
But this here is excellent value, if they offer it as part of their subscription coding plan. Paying by token could really add up. I did about 20 minutes of work and it cost me $1.50USD, and it's more expensive than Kimi 2.5.
Still 1/10th the cost of Opus 4.5 or Opus 4.6 when paying by the token.
Care to elaborate more?
Codex was super slow till 5.2 codex. Claude models were noticeably faster.