Kai-Fu Lee, the pc scientist recognized within the West for his bestseller AI Superpowers and in China for his bets on synthetic intelligence unicorns, has a brand new enterprise — and an amazing ambition.
In late March, Lee launched an organization known as 01.AI with the imaginative and prescient to develop a homegrown massive language mannequin for the Chinese language market. The enterprise places him in competitors with different outstanding Chinese language tech leaders, together with Sogou’s founder Wang Xiaochuan, who’ve been swiftly gathering expertise and enterprise capital to determine China’s equivalents of OpenAI.
“I feel necessity is the mom of innovation, and there’s clearly an enormous necessity in China,” Lee instructed TechCrunch in an interview, explaining the motive behind beginning 01.AI. “In contrast to the remainder of the world, China doesn’t have entry to OpenAI and Google as a result of these two firms didn’t make their merchandise obtainable in China, so I feel many doing LLM are attempting to do their half in creating an answer for a market that actually wants this.”
01.AI’s development is a becoming reflection of the fast growth within the generative AI subject. Seven months after its founding, the startup has launched its first mannequin, the open-source Yi-34B. The choice to introduce an open LLM as its debut product is a method to “give again” to society, stated Lee. For individuals who have felt LLaMA is a “godsend” to them, “we’ve supplied a compelling different,” he added.
As of writing, Yi-34B, which is a bilingual (English and Chinese language) base mannequin educated with 34 billion parameters and considerably smaller than different open fashions like Falcon-180B and Meta LlaMa2-70B, got here in first amongst pre-trained LLM fashions, according to a ranking by Hugging Face.
“We nonetheless imagine that bigger fashions, when educated properly, on a considerable amount of high-quality information, will all the time outperform considerably smaller fashions of comparable high quality and comparable expertise, so I feel [Yi-34B] outperforming a lot bigger fashions is one thing that we don’t normally see,” stated Lee. “We really feel fairly assured as we launched fashions which can be 100 billion to 400 billion over the following coming 12 months, 12 months and a half, these fashions will likely be dramatically higher than right now’s mannequin that we introduced.”
The startup’s potential to begin mannequin coaching rapidly is little question an final result of its easy fundraising, which is crucial to securing top-tier expertise and AI processors. Whereas declining to reveal how a lot 01.AI has raised, Lee stated it’s valued at $1 billion after receiving financing from Sinovation Ventures, Alibaba Cloud and different undisclosed buyers.
01.AI has already grown to greater than 100 workers, over half of whom are LLM consultants from main multinational and Chinese language tech companies. Its vp of expertise, as an illustration, is an early member of Google’s Bard, and its chief architect was a founding member of TensorFlow and labored alongside famend researchers like Jeff Dean and Samy Bengio at Google Mind. The important thing figures behind Yi-34B are Wenhao Huang, a Microsoft Analysis Asia veteran, and Ethan Dai, who held senior AI positions at Huawei and Alibaba.
Having backed over ten unicorns and venture-built seven firms by way of Sinovation Ventures, Lee is probably one of the well-connected buyers and entrepreneurs in China.
“It’s been, you recognize, over 25 years for the reason that founding of Microsoft Analysis Asia, and the whole lot I’ve completed has been about getting tremendous nice expertise,” stated Lee, who launched Microsoft Analysis Asia, the U.S. big’s largest analysis middle overseas, earlier than heading Google China. Through the years, Microsoft Analysis Asia has earned the popularity because the “West Level” for nurturing China’s AI entrepreneurs.
“Now, after all, you wish to pay folks pretty, and it is advisable be aggressive in pay, however I actually suppose that it’s additionally about folks believing they will make a distinction and believing the corporate can succeed,” Lee added.
It’s no secret that constructing LLMs is a pricey enterprise. To maintain its cash-intensive operations, 01.AI has plans for monetization proper from the beginning. Whereas the corporate will proceed to open supply a few of its fashions, its goal is to construct a state-of-the-art proprietary mannequin that serves as a basis for a various vary of economic merchandise.
“We can’t open supply the whole lot,” stated Lee. “We have been fairly cognizant of the truth that these massive language fashions require numerous compute, and subsequently, are very costly. Once we increase some huge cash, most of will probably be spent on the GPU. Provided that, we would have liked to first purchase as a lot GPU as we might, which we did.”
Like different LLM gamers in China, 01.AI has proactively stockpiled GPUs in anticipation of U.S. sanctions; it borrowed cash to purchase processors even earlier than it landed funding. Over the previous 12 months, the Biden administration has heightened restrictions on China’s entry to high-end AI chips, prompting Chinese firms to pay inflated prices for chips. The foresight was rewarded — 01.AI now has a provide that can suffice for a minimum of the following 12-18 months.
Other than inflicting complications for Chinese language companies, U.S. sanctions have been a catalyst for innovation by encouraging them to optimize using computing energy. “With a really high-quality infrastructure group, for each 1000 GPUs, we’d be capable of squeeze 2000 GPUs workload out of them,” stated Lee.
01.AI’s path to monetization hinges largely on its potential to search out product-market match for its costly AI fashions. Whereas top-notch LLM scientists are scarce, there’s no scarcity of product expertise in China.
“China’s not forward of the U.S. in LLM, however there’s little question China can construct higher functions than American builders largely due to the outstanding cellular web ecosystem that was constructed during the last 12 years or so,” argued Lee.
Whereas the founder gave no particulars on the providers within the pipeline, he hinted that the corporate is experimenting with ideas within the productiveness and social instructions, and he’d be “dissatisfied” if 01.AI didn’t launch an app inside this calendar 12 months.
The startup’s final aim, in keeping with Lee, is to turn out to be an ecosystem the place exterior builders can construct functions simply. “The obligation is not only to push out good analysis fashions, however much more importantly to make software growth straightforward in order that there will be compelling functions,” he stated. “On the finish of the day. It’s an ecosystem play.” Time will inform if Lee’s AI endeavor will repay.