Large Model

To lower the computional cost of training large model, we focus on speeding up pre-training by progressively growing from a small Transformer structure to a large one.

As scaling laws underscore the potential of increasing model sizes, the academic community has intensified its investigations into LLMs with capacities exceeding 50 billion parameters. This technical report builds on our prior work with Tele-FLM (also known as FLM-2), a publicly available 52-billion-parameter model.

FLM 系列大模型是 Cofe-AI 团队牵头完成的一系列大模型，包含FLM-2 (Tele-FLM-1T, Tele-FLM-52B, FLM-2-52B-Instruct等)，FLM-101B 和 FreeLM。其核心技术主要为生长技术，损失预测技术和 FreeLM 框架等。

我们构建并开源了阿拉伯语语言大模型 (ALM 1.0)。