The debate between open-source and closed-source models continues into the era of large-scale AI models.

Proponents of open-source argue that it fosters collaboration among global researchers and developers, accelerating the iteration and innovation of AI technology. Open source also allows more people to participate in improving and optimizing models and solving complex technical problems through collective intelligence.

On the other hand, advocates for proprietary models emphasize the advantages in commercialization, technical protection, and product differentiation. Proprietary models enable companies to control the pace of product development and market strategy, safeguarding their commercial interests. Moreover, keeping a model closed helps maintain a company’s technical edge, preventing competitors from imitating or surpassing them.

The release of Llama 3 has given open-source large models an edge in the competition with proprietary ones. Test results indicate that Llama 3 significantly outperforms Llama 2 and even surpasses GPT-3.5.

Meta continues to bet on open-source

Recently, Meta launched the open-source large model Llama 3, releasing both 8B and 70B versions. On the same day, Meta CEO Mark Zuckerberg announced that, based on Llama 3, Meta’s AI assistant now covers all its applications including Instagram, WhatsApp, Facebook, and it has also launched a dedicated website.

Compared to Llama 2, Llama 3 has undergone several key improvements: it uses a tokenizer with a 128K token vocabulary, which encodes language more effectively, thereby significantly enhancing model performance; both the 8B and 70B models employ Grouped Query Attention (GQA) to improve the inference efficiency of the Llama 3 model; the model is trained on sequences of 8192 tokens, using masks to ensure self-attention does not cross document boundaries.

According to Meta, Llama 3 has demonstrated state-of-the-art performance in various industry benchmark tests, offering new features including improved inference capabilities, making it the best open-source large model currently on the market.

Furthermore, following the release of Llama 3, Microsoft Azure, Google Cloud, Baidu Intelligent Cloud, and others have successively announced the integration of Llama 3 on their platforms. Baidu previously told the National Business Daily that Baidu Intelligent Cloud’s Qianfan large model platform is the first in China to launch a training and inference solution for the full range of Llama 3, facilitating developers to retrain and build their own large models.

Li Mingshun, Executive Director of the AI Application Working Group at the Ministry of Industry and Information Technology’s Industrial Culture Development Center, said in an interview with NBD that the release of Meta's Llama 3 could bring more AI application scenarios. By providing a larger vocabulary Token dictionary, longer context input, and optimized model structure, it enhances the model’s encoding efficiency and inference efficiency. “Now, compared to Llama 2, Llama 3 has significantly improved in coding ability and logical reasoning, which may promote the performance of related AI applications, especially in scenarios requiring complex logic and code understanding.”

After GPT-2, OpenAI turned its course towards proprietary models, while Meta has become a leader in the open-source community.

From the release of GPT-3 in 2020, to the sensational GPT-3.5, and the GPT-4 in March 2023, all are proprietary models. Previously, when Musk sued OpenAI, he bluntly said, “If OpenAI were renamed ClosedAI, I would withdraw the lawsuit.”

Li Mingshun believes that OpenAI’s shift may be related to its business strategy and market positioning. “Initially, open source helped to quickly attract attention and community participation. But as the company developed, I estimate that Sam Altman’s ambitions and business dreams have inflated, and going proprietary allows it to better finance, including cooperation with Microsoft, protecting its leading technological edge.” He said that OpenAI’s transformation also shows that the choice between open source and proprietary is not fixed but needs to be flexibly adjusted according to the company’s strategic goals, market environment, and product development stage. “Nothing in the world is absolute; many open-source companies also open-source suboptimal code while keeping their secret recipes proprietary.”

In the pursuit of AGI (Artificial General Intelligence), Meta continues to follow the open-source path to this day.

At the beginning of 2024, Zuckerberg stated in Meta’s fourth-quarter and full-year 2023 earnings call, “For a long time, our strategy has been to build and open source general infrastructure while keeping our specific product implementations proprietary.”

He believes that open source brings several strategic benefits. First, open-source software is usually more secure and reliable and more efficient due to the community’s continuous feedback, review, and development. Second, open-source software often becomes an industry standard, “When companies start to establish standards based on our technology stack, it makes it easier to integrate new innovations into our products. This subtle advantage, being able to learn and improve quickly, is a huge competitive edge, and becoming an industry standard is key to this capability.” Third, open source is extremely popular among developers and researchers.

Open-source and closed-source are not contradictory

The long-standing debate among global AI enthusiasts is whether to adopt an open-source or proprietary approach. Just days before the birth of Llama 3, a similar “debate” was taking place domestically.

On April 11th, Baidu’s Chairman and CEO, Robin Li, stated in an internal speech that there is little significance in open-sourcing large models, as proprietary models will continue to lead in capability, not just temporarily but sustainably. He also mentioned that the “dual-wheel drive” of startups both creating models and applications is not a good model. A week later, at the Create 2024 Baidu AI Developer Conference, Li reiterated, “People used to think open source was cheap, but in the context of large models, open source is the most expensive, so open-source models will fall increasingly behind.”

Zhou Hongyi, founder of Qihoo 360, holds a different view. “Some celebrities online talk nonsense, don’t be fooled by them saying that open source is not as good as proprietary. In a word, without open source, there would be no Linux (operating system kernel), no internet, and even the companies that say this have grown to where they are today by leveraging the power of open source.”

This statement was interpreted as a rebuttal to Robin Li’s views, which Zhou later clarified: “I have always been a believer in open source, but when I said open source is good, it was on April 13th at Harvard. Director Li (referring to Robin Li) said proprietary is good on April 16th in Beijing.”

Fang Han, chairman of Kunlun Tech, believes that the gap between open-source and proprietary models is narrowing. On April 16th, during an interview with NBD and other media, Fang stated: “The gap has evolved from being more than two years behind to being only 4-6 months behind from 2023 to this year.”

Moreover, Fang thinks that on the application front, proprietary lags behind open-source large models in terms of product features and satisfying long-tail needs. Open-source large models are ecosystem builders, better suited to meet users' long-tail demands. "I believe that open-source large models and commercial large models are parts of an ecosystem, not one overpowering the other. Everyone has their own space to exist."

In the debate between open source and proprietary, the answer may not be either/or. The future of AI might not be a completely open “free port” or a completely closed “island,” but a “mixed ecology” that includes both open collaboration and closed competition. In this ecosystem, open and closed are not opposing poles but two sides of the same coin.