Stay informed with free updates
Simply sign up for the Artificial intelligence myFT Digest – delivered straight to your inbox.
China’s efforts to match the United States’ computing power in artificial intelligence are being hampered by bug-ridden software, with customers of leading AI chipmaker Huawei complaining of performance issues and difficulty switching from Nvidia products.
The Chinese tech giant has emerged as the favorite in the race to develop a domestic alternative to industry leader Nvidia, after Washington further tightened export controls on high-performance silicon last October.
Its Ascend series has become an increasingly popular option for Chinese people AI groups to perform inference, a process that applications such as OpenAI’s ChatGPT use to generate answers to queries.
But several industry experts, including an AI engineer at a partner company, said the chips still lag far behind Nvidia’s in initial model training, blaming stability issues, slower interchip connectivity and subpar software developed by Huawei called Cann.
Nvidia’s Cuda software platform is known as the company’s “secret sauce” because it’s easy for developers to use and can dramatically speed up data processing. Huawei is one of several companies trying to break Nvidia’s stranglehold on AI chips by creating alternative software.
Huawei employees themselves are among those complaining about Cann. One researcher, who spoke on condition of anonymity, said it made Ascend’s product “difficult and unstable to use” and hampered testing work.
“When random errors occur, it’s very difficult to know where they’re coming from because of poor documentation. It takes talented developers to read the source code to see what the problem is, which slows everything down. The coding is imperfect,” they said.
Another Chinese engineer briefed on Baidu’s use of Huawei processors said the chips crashed frequently, complicating AI development work.
According to the Huawei researcher, the failures occurred because the hardware was difficult to use. “It is easy to get bad results because people are not familiar with the hardware itself,” they said.
To address the issue, Huawei has dispatched engineers to help customers on-site transfer training code previously written on Cuda to Cann, multiple people familiar with the matter said. Baidu, iFlytek and Tencent are among the tech companies that have received teams of engineers, the people said.
Huawei declined to comment. Baidu, iFlytek and Tencent did not respond to requests for comment.
A former Baidu employee said: “Huawei is very good at customer service, which is why they of course have engineers on site at their big customers, to help them use their chips.”
Huawei can rely on a large workforce to accelerate this transition. According to the company, more than 50% of its 207,000 employees work in research and development, including engineers sent to install technologies at customers’ sites.
“Huawei’s advantage over Nvidia is that it can work closely with its customers,” said Tilly Zhang, a technology analyst at consultancy Gavekal. “Unlike Nvidia, Huawei has a large team of engineers to help customers solve problems and transition to their hardware.”
Huawei has also set up an online portal for developers to provide feedback on how their software can be improved.
After the United States Export controls to be tightened in OctoberHuawei has raised the price of its Ascend 910B training chip by 20% to 30%, people familiar with the matter said.
Huawei customers have also expressed concern about supply constraints for the Ascend chip, likely due to manufacturing difficulties, with Chinese companies being blocked from purchasing advanced chipmaking machines from Dutch firm ASML.
Huawei has seen strong demand for its AI chips. The group reported a 34% rise in first-half revenue on Thursday, without providing details on sales from its various businesses.
More than 50 fundamental models have been “trained and iterated” on the Ascend chip, Huawei CEO Zhang Ping’an said at the World Conference on Artificial Intelligence in Shanghai in July.
Iflytek said its large language model was trained exclusively on Huawei chips after Huawei sent a group of engineers to its headquarters in Hefei, eastern China, last year to integrate the technology.