Baidu Releases 280-Billion-Parameter ERNIE 5.5 Flash Base as Open Model
The Chinese technology company Baidu has released ERNIE 5.5 Flash Base, a base model with 280 billion parameters, as an open-source model that is delivered without fine-tuning.
The Chinese technology company Baidu has released ERNIE 5.5 Flash Base, a large language model, as an open-source base model. The model comprises a total of 280 billion parameters and, according to the AI news portal The Decoder, was delivered without fine-tuning. Developers can therefore freely adapt it to their own applications without first being bound to any specialization made by Baidu.
Technically, ERNIE 5.5 Flash Base relies on a Mixture-of-Experts architecture. During processing, not all parts of the network are activated at the same time; instead, specialized sub-models, known as experts, are selected depending on the input. This design is considered a widespread approach in the development of large models for increasing the overall size of a system without raising the computational effort of each individual request to the same extent. The total size of 280 billion parameters describes the full scope of the model, not the number of parameters that are actually active for a single request.
The model was pretrained on 33.5 trillion tokens. Tokens are the smallest text units with which language models work, and this figure indicates how extensive the training data was. Releasing the model as a base model means that the system initially forms the general foundation on which other developers can build, for example for specific industries, tasks or languages. The delivery without fine-tuning in particular distinguishes this version from models that have already been optimized for specific chat or assistant applications.
The move fits into a broader trend in which Chinese and Western developers increasingly make large models available as open weights. For companies, research institutions and independent developers, such a base model can ease access to powerful AI, since their own training costs are eliminated or at least reduced. At the same time, use is only as free as the license permits, and the terms for this were not listed in the material available.
It therefore remains open how ERNIE 5.5 Flash Base compares with competing open models. The report on which this news is based contains no benchmark results, and further model details such as context length or areas of application were not mentioned in the available material. Whether and to what extent the model will be used in practice cannot therefore yet be assessed from the information available.
Source: the-decoder.com
14
